06 August, 2021

Motivating data

load(file = "data/manipulationDatasets.RData")

dat.1

Treatment Plot Dose Time Resp1 Resp2
Control P1 H 1 8.12 3.06
Control P1 H 2 20.55 25.94
Control P1 H 3 27.49 29.85
Control P1 H 4 44.79 25.39
Control P1 M 1 20.99 20.31
Control P1 M 2 37.54 17.62
Control P1 M 3 61.46 98.44
Control P1 M 4 82.21 160
Control P1 L 1 31.73 21.22
Control P1 L 2 59.08 37.51
Control P1 L 3 94.54 119.2
Control P1 L 4 121.2 116.5
Control P2 H 1 8.14 23.93
Control P2 H 2 13.36 28.02
Control P2 H 3 33.37 37.17
Control P2 H 4 39.87 38.25
Control P2 M 1 19.95 19.73
Control P2 M 2 42.83 40.52
Control P2 M 3 62.46 4.81
Control P2 M 4 81.78 136.7
Control P2 L 1 32.76 30.7
Control P2 L 2 62.35 123.8
Control P2 L 3 90.22 113.9
Control P2 L 4 114 76.52
Exclusion P3 H 1 21.86 23.58
Exclusion P3 H 2 39.83 28.03
Exclusion P3 H 3 59.53 21.32
Exclusion P3 H 4 75.59 90.76
Exclusion P3 M 1 38.57 30.63
Exclusion P3 M 2 81.25 83.61
Exclusion P3 M 3 124.1 124.1
Exclusion P3 M 4 159.7 112.7
Exclusion P3 L 1 61.16 39.53
Exclusion P3 L 2 119.8 110.3
Exclusion P3 L 3 175.9 286.3
Exclusion P3 L 4 238.8 54.23
Exclusion P4 H 1 18.82 28.6
Exclusion P4 H 2 39.82 39.07
Exclusion P4 H 3 63.3 93.43
Exclusion P4 H 4 82.29 60.15
Exclusion P4 M 1 39.51 45.9
Exclusion P4 M 2 79.24 88.04
Exclusion P4 M 3 122.1 84.19
Exclusion P4 M 4 161.7 256.3
Exclusion P4 L 1 57.93 85.24
Exclusion P4 L 2 117.9 167.9
Exclusion P4 L 3 181.1 314.5
Exclusion P4 L 4 242.3 304.7

Data manipulation

  • base R syntax provides ways to manipulate and access a variety of data structures
  • code can be difficult to read and keep organised
  • exacerbated for grouped data
Treatment Plot Dose Time Resp1 Resp2
Control P1 H 1 8.12 3.06
Control P1 H 2 20.55 25.94
Control P1 H 3 27.49 29.85
Control P1 H 4 44.79 25.39
Control P1 M 1 20.99 20.31
Control P1 M 2 37.54 17.62
Control P1 M 3 61.46 98.44
Control P1 M 4 82.21 160
Control P1 L 1 31.73 21.22
Control P1 L 2 59.08 37.51
Control P1 L 3 94.54 119.2
Control P1 L 4 121.2 116.5
Control P2 H 1 8.14 23.93
Control P2 H 2 13.36 28.02
Control P2 H 3 33.37 37.17
Control P2 H 4 39.87 38.25
Control P2 M 1 19.95 19.73
Control P2 M 2 42.83 40.52
Control P2 M 3 62.46 4.81
Control P2 M 4 81.78 136.7
Control P2 L 1 32.76 30.7
Control P2 L 2 62.35 123.8
Control P2 L 3 90.22 113.9
Control P2 L 4 114 76.52
Exclusion P3 H 1 21.86 23.58
Exclusion P3 H 2 39.83 28.03
Exclusion P3 H 3 59.53 21.32
Exclusion P3 H 4 75.59 90.76
Exclusion P3 M 1 38.57 30.63
Exclusion P3 M 2 81.25 83.61
Exclusion P3 M 3 124.1 124.1
Exclusion P3 M 4 159.7 112.7
Exclusion P3 L 1 61.16 39.53
Exclusion P3 L 2 119.8 110.3
Exclusion P3 L 3 175.9 286.3
Exclusion P3 L 4 238.8 54.23
Exclusion P4 H 1 18.82 28.6
Exclusion P4 H 2 39.82 39.07
Exclusion P4 H 3 63.3 93.43
Exclusion P4 H 4 82.29 60.15
Exclusion P4 M 1 39.51 45.9
Exclusion P4 M 2 79.24 88.04
Exclusion P4 M 3 122.1 84.19
Exclusion P4 M 4 161.7 256.3
Exclusion P4 L 1 57.93 85.24
Exclusion P4 L 2 117.9 167.9
Exclusion P4 L 3 181.1 314.5
Exclusion P4 L 4 242.3 304.7

Data manipulation

  • enter the tidyverse ecosystem
  • a coherent set of manipulation focussed packages

Data manipulation

Manipulation functions (verbs)

Task Function Package
Sorting arrange() dplyr
Adding columns mutate() dplyr
Transformations mutate() dplyr
Re-ordering factor levels factor(,levels=) base
Re-labelling factor(,lab=) base
recode() dplyr
Re-naming columns rename(,replace=) dplyr
Filtering/Subsetting indexing base
~ columns select(,...) dplyr
pull(,...) dplyr
~ rows filter(,...) dplyr
Unique combinations distinct() dplyr
Reshaping data pivot_longer(), pivot_wider() tidyr
Split/combine columns separate(), unite() tidyr
Aggregating group_by() summarise() dplyr
group_by() count() dplyr
Merging/joining *_join() dplyr
Extracting data structure expand() tidyr
crossing() tidyr

Data manipulation grammar

Piping

  • %>%
data %>%
    select(...) %>%
    group_by(...) %>%
    summarise(...)


Any object can be passed (piped) to any function if that object is normally the function’s first argument

Motivating data

load(file = "../data/manipulationDatasets.RData")

dat.1

Treatment Plot Dose Time Resp1 Resp2
Control P1 H 1 8.12 3.06
Control P1 H 2 20.55 25.94
Control P1 H 3 27.49 29.85
Control P1 H 4 44.79 25.39
Control P1 M 1 20.99 20.31
Control P1 M 2 37.54 17.62
Control P1 M 3 61.46 98.44
Control P1 M 4 82.21 160
Control P1 L 1 31.73 21.22
Control P1 L 2 59.08 37.51
Control P1 L 3 94.54 119.2
Control P1 L 4 121.2 116.5
Control P2 H 1 8.14 23.93
Control P2 H 2 13.36 28.02
Control P2 H 3 33.37 37.17
Control P2 H 4 39.87 38.25
Control P2 M 1 19.95 19.73
Control P2 M 2 42.83 40.52
Control P2 M 3 62.46 4.81
Control P2 M 4 81.78 136.7
Control P2 L 1 32.76 30.7
Control P2 L 2 62.35 123.8
Control P2 L 3 90.22 113.9
Control P2 L 4 114 76.52
Exclusion P3 H 1 21.86 23.58
Exclusion P3 H 2 39.83 28.03
Exclusion P3 H 3 59.53 21.32
Exclusion P3 H 4 75.59 90.76
Exclusion P3 M 1 38.57 30.63
Exclusion P3 M 2 81.25 83.61
Exclusion P3 M 3 124.1 124.1
Exclusion P3 M 4 159.7 112.7
Exclusion P3 L 1 61.16 39.53
Exclusion P3 L 2 119.8 110.3
Exclusion P3 L 3 175.9 286.3
Exclusion P3 L 4 238.8 54.23
Exclusion P4 H 1 18.82 28.6
Exclusion P4 H 2 39.82 39.07
Exclusion P4 H 3 63.3 93.43
Exclusion P4 H 4 82.29 60.15
Exclusion P4 M 1 39.51 45.9
Exclusion P4 M 2 79.24 88.04
Exclusion P4 M 3 122.1 84.19
Exclusion P4 M 4 161.7 256.3
Exclusion P4 L 1 57.93 85.24
Exclusion P4 L 2 117.9 167.9
Exclusion P4 L 3 181.1 314.5
Exclusion P4 L 4 242.3 304.7

Manipulating data

  • re(arranging) data
  • subsetting data
  • summarising/aggregating data
  • joining datasets
Treatment Plot Dose Time Resp1 Resp2
Control P1 H 1 8.12 3.06
Control P1 H 2 20.55 25.94
Control P1 H 3 27.49 29.85
Control P1 H 4 44.79 25.39
Control P1 M 1 20.99 20.31
Control P1 M 2 37.54 17.62
Control P1 M 3 61.46 98.44
Control P1 M 4 82.21 160
Control P1 L 1 31.73 21.22
Control P1 L 2 59.08 37.51
Control P1 L 3 94.54 119.2
Control P1 L 4 121.2 116.5
Control P2 H 1 8.14 23.93
Control P2 H 2 13.36 28.02
Control P2 H 3 33.37 37.17
Control P2 H 4 39.87 38.25
Control P2 M 1 19.95 19.73
Control P2 M 2 42.83 40.52
Control P2 M 3 62.46 4.81
Control P2 M 4 81.78 136.7
Control P2 L 1 32.76 30.7
Control P2 L 2 62.35 123.8
Control P2 L 3 90.22 113.9
Control P2 L 4 114 76.52
Exclusion P3 H 1 21.86 23.58
Exclusion P3 H 2 39.83 28.03
Exclusion P3 H 3 59.53 21.32
Exclusion P3 H 4 75.59 90.76
Exclusion P3 M 1 38.57 30.63
Exclusion P3 M 2 81.25 83.61
Exclusion P3 M 3 124.1 124.1
Exclusion P3 M 4 159.7 112.7
Exclusion P3 L 1 61.16 39.53
Exclusion P3 L 2 119.8 110.3
Exclusion P3 L 3 175.9 286.3
Exclusion P3 L 4 238.8 54.23
Exclusion P4 H 1 18.82 28.6
Exclusion P4 H 2 39.82 39.07
Exclusion P4 H 3 63.3 93.43
Exclusion P4 H 4 82.29 60.15
Exclusion P4 M 1 39.51 45.9
Exclusion P4 M 2 79.24 88.04
Exclusion P4 M 3 122.1 84.19
Exclusion P4 M 4 161.7 256.3
Exclusion P4 L 1 57.93 85.24
Exclusion P4 L 2 117.9 167.9
Exclusion P4 L 3 181.1 314.5
Exclusion P4 L 4 242.3 304.7

Data manipulation packages

library(dplyr)
library(tidyr)
#OR to get the entire ecosystem
library(tidyverse)

Data files

head(dat.1)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94
## 3   Control   P1    H    3 27.49 29.85
## 4   Control   P1    H    4 44.79 25.39
## 5   Control   P1    M    1 20.99 20.31
## 6   Control   P1    M    2 37.54 17.62
#OR
dat.1 %>% head()
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94
## 3   Control   P1    H    3 27.49 29.85
## 4   Control   P1    H    4 44.79 25.39
## 5   Control   P1    M    1 20.99 20.31
## 6   Control   P1    M    2 37.54 17.62
#OR
dat.1 %>% head
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94
## 3   Control   P1    H    3 27.49 29.85
## 4   Control   P1    H    4 44.79 25.39
## 5   Control   P1    M    1 20.99 20.31
## 6   Control   P1    M    2 37.54 17.62

Data files

summary(dat.1)
##      Treatment  Plot    Dose        Time          Resp1            Resp2       
##  Control  :24   P1:12   H:16   Min.   :1.00   Min.   :  8.12   Min.   :  3.06  
##  Exclusion:24   P2:12   L:16   1st Qu.:1.75   1st Qu.: 36.50   1st Qu.: 28.03  
##                 P3:12   M:16   Median :2.50   Median : 61.31   Median : 50.06  
##                 P4:12          Mean   :2.50   Mean   : 75.27   Mean   : 81.71  
##                                3rd Qu.:3.25   3rd Qu.: 99.41   3rd Qu.:112.95  
##                                Max.   :4.00   Max.   :242.31   Max.   :314.49

Data files

summary(dat.1)
##      Treatment  Plot    Dose        Time          Resp1            Resp2       
##  Control  :24   P1:12   H:16   Min.   :1.00   Min.   :  8.12   Min.   :  3.06  
##  Exclusion:24   P2:12   L:16   1st Qu.:1.75   1st Qu.: 36.50   1st Qu.: 28.03  
##                 P3:12   M:16   Median :2.50   Median : 61.31   Median : 50.06  
##                 P4:12          Mean   :2.50   Mean   : 75.27   Mean   : 81.71  
##                                3rd Qu.:3.25   3rd Qu.: 99.41   3rd Qu.:112.95  
##                                Max.   :4.00   Max.   :242.31   Max.   :314.49
dat.1 %>% summary()
##      Treatment  Plot    Dose        Time          Resp1            Resp2       
##  Control  :24   P1:12   H:16   Min.   :1.00   Min.   :  8.12   Min.   :  3.06  
##  Exclusion:24   P2:12   L:16   1st Qu.:1.75   1st Qu.: 36.50   1st Qu.: 28.03  
##                 P3:12   M:16   Median :2.50   Median : 61.31   Median : 50.06  
##                 P4:12          Mean   :2.50   Mean   : 75.27   Mean   : 81.71  
##                                3rd Qu.:3.25   3rd Qu.: 99.41   3rd Qu.:112.95  
##                                Max.   :4.00   Max.   :242.31   Max.   :314.49
dat.1 %>% summary
##      Treatment  Plot    Dose        Time          Resp1            Resp2       
##  Control  :24   P1:12   H:16   Min.   :1.00   Min.   :  8.12   Min.   :  3.06  
##  Exclusion:24   P2:12   L:16   1st Qu.:1.75   1st Qu.: 36.50   1st Qu.: 28.03  
##                 P3:12   M:16   Median :2.50   Median : 61.31   Median : 50.06  
##                 P4:12          Mean   :2.50   Mean   : 75.27   Mean   : 81.71  
##                                3rd Qu.:3.25   3rd Qu.: 99.41   3rd Qu.:112.95  
##                                Max.   :4.00   Max.   :242.31   Max.   :314.49

Data files

str(dat.1)
## 'data.frame':    48 obs. of  6 variables:
##  $ Treatment: Factor w/ 2 levels "Control","Exclusion": 1 1 1 1 1 1 1 1 1 1 ...
##  $ Plot     : Factor w/ 4 levels "P1","P2","P3",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ Dose     : Factor w/ 3 levels "H","L","M": 1 1 1 1 3 3 3 3 2 2 ...
##  $ Time     : int  1 2 3 4 1 2 3 4 1 2 ...
##  $ Resp1    : num  8.12 20.55 27.49 44.79 20.99 ...
##  $ Resp2    : num  3.06 25.94 29.85 25.39 20.31 ...

Dense summary

glimpse(dat.1)
## Rows: 48
## Columns: 6
## $ Treatment <fct> Control, Control, Control, Control, Control, Control, Control, Control…
## $ Plot      <fct> P1, P1, P1, P1, P1, P1, P1, P1, P1, P1, P1, P1, P2, P2, P2, P2, P2, P2…
## $ Dose      <fct> H, H, H, H, M, M, M, M, L, L, L, L, H, H, H, H, M, M, M, M, L, L, L, L…
## $ Time      <int> 1, 2, 3, 4, 1, 2, 3, 4, 1, 2, 3, 4, 1, 2, 3, 4, 1, 2, 3, 4, 1, 2, 3, 4…
## $ Resp1     <dbl> 8.12, 20.55, 27.49, 44.79, 20.99, 37.54, 61.46, 82.21, 31.73, 59.08, 9…
## $ Resp2     <dbl> 3.06, 25.94, 29.85, 25.39, 20.31, 17.62, 98.44, 160.01, 21.22, 37.51, …

Dataframes and tibbles

Dataframes and tibbles

dat.1 %>% as_tibble
## # A tibble: 48 x 6
##    Treatment Plot  Dose   Time Resp1  Resp2
##    <fct>     <fct> <fct> <int> <dbl>  <dbl>
##  1 Control   P1    H         1  8.12   3.06
##  2 Control   P1    H         2 20.6   25.9 
##  3 Control   P1    H         3 27.5   29.8 
##  4 Control   P1    H         4 44.8   25.4 
##  5 Control   P1    M         1 21.0   20.3 
##  6 Control   P1    M         2 37.5   17.6 
##  7 Control   P1    M         3 61.5   98.4 
##  8 Control   P1    M         4 82.2  160.  
##  9 Control   P1    L         1 31.7   21.2 
## 10 Control   P1    L         2 59.1   37.5 
## # … with 38 more rows

Sorting data




Sorting data (arrange)

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94

Sorting by Resp1

dat.1 %>% arrange(Resp1)
##    Treatment Plot Dose Time  Resp1  Resp2
## 1    Control   P1    H    1   8.12   3.06
## 2    Control   P2    H    1   8.14  23.93
## 3    Control   P2    H    2  13.36  28.02
## 4  Exclusion   P4    H    1  18.82  28.60
## 5    Control   P2    M    1  19.95  19.73
## 6    Control   P1    H    2  20.55  25.94
## 7    Control   P1    M    1  20.99  20.31
## 8  Exclusion   P3    H    1  21.86  23.58
## 9    Control   P1    H    3  27.49  29.85
## 10   Control   P1    L    1  31.73  21.22
## 11   Control   P2    L    1  32.76  30.70
## 12   Control   P2    H    3  33.37  37.17
## 13   Control   P1    M    2  37.54  17.62
## 14 Exclusion   P3    M    1  38.57  30.63
## 15 Exclusion   P4    M    1  39.51  45.90
## 16 Exclusion   P4    H    2  39.82  39.07
## 17 Exclusion   P3    H    2  39.83  28.03
## 18   Control   P2    H    4  39.87  38.25
## 19   Control   P2    M    2  42.83  40.52
## 20   Control   P1    H    4  44.79  25.39
## 21 Exclusion   P4    L    1  57.93  85.24
## 22   Control   P1    L    2  59.08  37.51
## 23 Exclusion   P3    H    3  59.53  21.32
## 24 Exclusion   P3    L    1  61.16  39.53
## 25   Control   P1    M    3  61.46  98.44
## 26   Control   P2    L    2  62.35 123.78
## 27   Control   P2    M    3  62.46   4.81
## 28 Exclusion   P4    H    3  63.30  93.43
## 29 Exclusion   P3    H    4  75.59  90.76
## 30 Exclusion   P4    M    2  79.24  88.04
## 31 Exclusion   P3    M    2  81.25  83.61
## 32   Control   P2    M    4  81.78 136.66
## 33   Control   P1    M    4  82.21 160.01
## 34 Exclusion   P4    H    4  82.29  60.15
## 35   Control   P2    L    3  90.22 113.87
## 36   Control   P1    L    3  94.54 119.22
## 37   Control   P2    L    4 114.03  76.52
## 38 Exclusion   P4    L    2 117.88 167.90
## 39 Exclusion   P3    L    2 119.84 110.27
## 40   Control   P1    L    4 121.17 116.45
## 41 Exclusion   P4    M    3 122.09  84.19
## 42 Exclusion   P3    M    3 124.08 124.09
## 43 Exclusion   P3    M    4 159.69 112.65
## 44 Exclusion   P4    M    4 161.67 256.34
## 45 Exclusion   P3    L    3 175.87 286.33
## 46 Exclusion   P4    L    3 181.09 314.49
## 47 Exclusion   P3    L    4 238.76  54.23
## 48 Exclusion   P4    L    4 242.31 304.70

Sorting data (arrange)

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94

Sorting by Resp1 (descending order)

dat.1 %>% arrange(-Resp1)
##    Treatment Plot Dose Time  Resp1  Resp2
## 1  Exclusion   P4    L    4 242.31 304.70
## 2  Exclusion   P3    L    4 238.76  54.23
## 3  Exclusion   P4    L    3 181.09 314.49
## 4  Exclusion   P3    L    3 175.87 286.33
## 5  Exclusion   P4    M    4 161.67 256.34
## 6  Exclusion   P3    M    4 159.69 112.65
## 7  Exclusion   P3    M    3 124.08 124.09
## 8  Exclusion   P4    M    3 122.09  84.19
## 9    Control   P1    L    4 121.17 116.45
## 10 Exclusion   P3    L    2 119.84 110.27
## 11 Exclusion   P4    L    2 117.88 167.90
## 12   Control   P2    L    4 114.03  76.52
## 13   Control   P1    L    3  94.54 119.22
## 14   Control   P2    L    3  90.22 113.87
## 15 Exclusion   P4    H    4  82.29  60.15
## 16   Control   P1    M    4  82.21 160.01
## 17   Control   P2    M    4  81.78 136.66
## 18 Exclusion   P3    M    2  81.25  83.61
## 19 Exclusion   P4    M    2  79.24  88.04
## 20 Exclusion   P3    H    4  75.59  90.76
## 21 Exclusion   P4    H    3  63.30  93.43
## 22   Control   P2    M    3  62.46   4.81
## 23   Control   P2    L    2  62.35 123.78
## 24   Control   P1    M    3  61.46  98.44
## 25 Exclusion   P3    L    1  61.16  39.53
## 26 Exclusion   P3    H    3  59.53  21.32
## 27   Control   P1    L    2  59.08  37.51
## 28 Exclusion   P4    L    1  57.93  85.24
## 29   Control   P1    H    4  44.79  25.39
## 30   Control   P2    M    2  42.83  40.52
## 31   Control   P2    H    4  39.87  38.25
## 32 Exclusion   P3    H    2  39.83  28.03
## 33 Exclusion   P4    H    2  39.82  39.07
## 34 Exclusion   P4    M    1  39.51  45.90
## 35 Exclusion   P3    M    1  38.57  30.63
## 36   Control   P1    M    2  37.54  17.62
## 37   Control   P2    H    3  33.37  37.17
## 38   Control   P2    L    1  32.76  30.70
## 39   Control   P1    L    1  31.73  21.22
## 40   Control   P1    H    3  27.49  29.85
## 41 Exclusion   P3    H    1  21.86  23.58
## 42   Control   P1    M    1  20.99  20.31
## 43   Control   P1    H    2  20.55  25.94
## 44   Control   P2    M    1  19.95  19.73
## 45 Exclusion   P4    H    1  18.82  28.60
## 46   Control   P2    H    2  13.36  28.02
## 47   Control   P2    H    1   8.14  23.93
## 48   Control   P1    H    1   8.12   3.06

Sorting data (arrange)

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94

Sorting by Dose and then Resp1

dat.1 %>% arrange(Dose,Resp1)
##    Treatment Plot Dose Time  Resp1  Resp2
## 1    Control   P1    H    1   8.12   3.06
## 2    Control   P2    H    1   8.14  23.93
## 3    Control   P2    H    2  13.36  28.02
## 4  Exclusion   P4    H    1  18.82  28.60
## 5    Control   P1    H    2  20.55  25.94
## 6  Exclusion   P3    H    1  21.86  23.58
## 7    Control   P1    H    3  27.49  29.85
## 8    Control   P2    H    3  33.37  37.17
## 9  Exclusion   P4    H    2  39.82  39.07
## 10 Exclusion   P3    H    2  39.83  28.03
## 11   Control   P2    H    4  39.87  38.25
## 12   Control   P1    H    4  44.79  25.39
## 13 Exclusion   P3    H    3  59.53  21.32
## 14 Exclusion   P4    H    3  63.30  93.43
## 15 Exclusion   P3    H    4  75.59  90.76
## 16 Exclusion   P4    H    4  82.29  60.15
## 17   Control   P1    L    1  31.73  21.22
## 18   Control   P2    L    1  32.76  30.70
## 19 Exclusion   P4    L    1  57.93  85.24
## 20   Control   P1    L    2  59.08  37.51
## 21 Exclusion   P3    L    1  61.16  39.53
## 22   Control   P2    L    2  62.35 123.78
## 23   Control   P2    L    3  90.22 113.87
## 24   Control   P1    L    3  94.54 119.22
## 25   Control   P2    L    4 114.03  76.52
## 26 Exclusion   P4    L    2 117.88 167.90
## 27 Exclusion   P3    L    2 119.84 110.27
## 28   Control   P1    L    4 121.17 116.45
## 29 Exclusion   P3    L    3 175.87 286.33
## 30 Exclusion   P4    L    3 181.09 314.49
## 31 Exclusion   P3    L    4 238.76  54.23
## 32 Exclusion   P4    L    4 242.31 304.70
## 33   Control   P2    M    1  19.95  19.73
## 34   Control   P1    M    1  20.99  20.31
## 35   Control   P1    M    2  37.54  17.62
## 36 Exclusion   P3    M    1  38.57  30.63
## 37 Exclusion   P4    M    1  39.51  45.90
## 38   Control   P2    M    2  42.83  40.52
## 39   Control   P1    M    3  61.46  98.44
## 40   Control   P2    M    3  62.46   4.81
## 41 Exclusion   P4    M    2  79.24  88.04
## 42 Exclusion   P3    M    2  81.25  83.61
## 43   Control   P2    M    4  81.78 136.66
## 44   Control   P1    M    4  82.21 160.01
## 45 Exclusion   P4    M    3 122.09  84.19
## 46 Exclusion   P3    M    3 124.08 124.09
## 47 Exclusion   P3    M    4 159.69 112.65
## 48 Exclusion   P4    M    4 161.67 256.34

Sorting data (arrange)

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94

Sort by the sum of Resp1 and Resp2

dat.1 %>% arrange(Resp1+Resp2)
##    Treatment Plot Dose Time  Resp1  Resp2
## 1    Control   P1    H    1   8.12   3.06
## 2    Control   P2    H    1   8.14  23.93
## 3    Control   P2    M    1  19.95  19.73
## 4    Control   P1    M    1  20.99  20.31
## 5    Control   P2    H    2  13.36  28.02
## 6  Exclusion   P3    H    1  21.86  23.58
## 7    Control   P1    H    2  20.55  25.94
## 8  Exclusion   P4    H    1  18.82  28.60
## 9    Control   P1    L    1  31.73  21.22
## 10   Control   P1    M    2  37.54  17.62
## 11   Control   P1    H    3  27.49  29.85
## 12   Control   P2    L    1  32.76  30.70
## 13   Control   P2    M    3  62.46   4.81
## 14 Exclusion   P3    H    2  39.83  28.03
## 15 Exclusion   P3    M    1  38.57  30.63
## 16   Control   P1    H    4  44.79  25.39
## 17   Control   P2    H    3  33.37  37.17
## 18   Control   P2    H    4  39.87  38.25
## 19 Exclusion   P4    H    2  39.82  39.07
## 20 Exclusion   P3    H    3  59.53  21.32
## 21   Control   P2    M    2  42.83  40.52
## 22 Exclusion   P4    M    1  39.51  45.90
## 23   Control   P1    L    2  59.08  37.51
## 24 Exclusion   P3    L    1  61.16  39.53
## 25 Exclusion   P4    H    4  82.29  60.15
## 26 Exclusion   P4    L    1  57.93  85.24
## 27 Exclusion   P4    H    3  63.30  93.43
## 28   Control   P1    M    3  61.46  98.44
## 29 Exclusion   P3    M    2  81.25  83.61
## 30 Exclusion   P3    H    4  75.59  90.76
## 31 Exclusion   P4    M    2  79.24  88.04
## 32   Control   P2    L    2  62.35 123.78
## 33   Control   P2    L    4 114.03  76.52
## 34   Control   P2    L    3  90.22 113.87
## 35 Exclusion   P4    M    3 122.09  84.19
## 36   Control   P1    L    3  94.54 119.22
## 37   Control   P2    M    4  81.78 136.66
## 38 Exclusion   P3    L    2 119.84 110.27
## 39   Control   P1    L    4 121.17 116.45
## 40   Control   P1    M    4  82.21 160.01
## 41 Exclusion   P3    M    3 124.08 124.09
## 42 Exclusion   P3    M    4 159.69 112.65
## 43 Exclusion   P4    L    2 117.88 167.90
## 44 Exclusion   P3    L    4 238.76  54.23
## 45 Exclusion   P4    M    4 161.67 256.34
## 46 Exclusion   P3    L    3 175.87 286.33
## 47 Exclusion   P4    L    3 181.09 314.49
## 48 Exclusion   P4    L    4 242.31 304.70

Your turn

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94
  • sort by Treatment and then Time

Your turn

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94
  • sort by Treatment and then Time
dat.1 %>% arrange(Treatment,Time)
##    Treatment Plot Dose Time  Resp1  Resp2
## 1    Control   P1    H    1   8.12   3.06
## 2    Control   P1    M    1  20.99  20.31
## 3    Control   P1    L    1  31.73  21.22
## 4    Control   P2    H    1   8.14  23.93
## 5    Control   P2    M    1  19.95  19.73
## 6    Control   P2    L    1  32.76  30.70
## 7    Control   P1    H    2  20.55  25.94
## 8    Control   P1    M    2  37.54  17.62
## 9    Control   P1    L    2  59.08  37.51
## 10   Control   P2    H    2  13.36  28.02
## 11   Control   P2    M    2  42.83  40.52
## 12   Control   P2    L    2  62.35 123.78
## 13   Control   P1    H    3  27.49  29.85
## 14   Control   P1    M    3  61.46  98.44
## 15   Control   P1    L    3  94.54 119.22
## 16   Control   P2    H    3  33.37  37.17
## 17   Control   P2    M    3  62.46   4.81
## 18   Control   P2    L    3  90.22 113.87
## 19   Control   P1    H    4  44.79  25.39
## 20   Control   P1    M    4  82.21 160.01
## 21   Control   P1    L    4 121.17 116.45
## 22   Control   P2    H    4  39.87  38.25
## 23   Control   P2    M    4  81.78 136.66
## 24   Control   P2    L    4 114.03  76.52
## 25 Exclusion   P3    H    1  21.86  23.58
## 26 Exclusion   P3    M    1  38.57  30.63
## 27 Exclusion   P3    L    1  61.16  39.53
## 28 Exclusion   P4    H    1  18.82  28.60
## 29 Exclusion   P4    M    1  39.51  45.90
## 30 Exclusion   P4    L    1  57.93  85.24
## 31 Exclusion   P3    H    2  39.83  28.03
## 32 Exclusion   P3    M    2  81.25  83.61
## 33 Exclusion   P3    L    2 119.84 110.27
## 34 Exclusion   P4    H    2  39.82  39.07
## 35 Exclusion   P4    M    2  79.24  88.04
## 36 Exclusion   P4    L    2 117.88 167.90
## 37 Exclusion   P3    H    3  59.53  21.32
## 38 Exclusion   P3    M    3 124.08 124.09
## 39 Exclusion   P3    L    3 175.87 286.33
## 40 Exclusion   P4    H    3  63.30  93.43
## 41 Exclusion   P4    M    3 122.09  84.19
## 42 Exclusion   P4    L    3 181.09 314.49
## 43 Exclusion   P3    H    4  75.59  90.76
## 44 Exclusion   P3    M    4 159.69 112.65
## 45 Exclusion   P3    L    4 238.76  54.23
## 46 Exclusion   P4    H    4  82.29  60.15
## 47 Exclusion   P4    M    4 161.67 256.34
## 48 Exclusion   P4    L    4 242.31 304.70

Your turn

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94
  • sort by Treatment and then the mean of Resp 1 and Resp2

Your turn

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94
  • sort by Treatment and then the mean of Resp1 and Resp2
dat.1 %>% arrange(Treatment, mean(c(Resp1, Resp2)))
##    Treatment Plot Dose Time  Resp1  Resp2
## 1    Control   P1    H    1   8.12   3.06
## 2    Control   P1    H    2  20.55  25.94
## 3    Control   P1    H    3  27.49  29.85
## 4    Control   P1    H    4  44.79  25.39
## 5    Control   P1    M    1  20.99  20.31
## 6    Control   P1    M    2  37.54  17.62
## 7    Control   P1    M    3  61.46  98.44
## 8    Control   P1    M    4  82.21 160.01
## 9    Control   P1    L    1  31.73  21.22
## 10   Control   P1    L    2  59.08  37.51
## 11   Control   P1    L    3  94.54 119.22
## 12   Control   P1    L    4 121.17 116.45
## 13   Control   P2    H    1   8.14  23.93
## 14   Control   P2    H    2  13.36  28.02
## 15   Control   P2    H    3  33.37  37.17
## 16   Control   P2    H    4  39.87  38.25
## 17   Control   P2    M    1  19.95  19.73
## 18   Control   P2    M    2  42.83  40.52
## 19   Control   P2    M    3  62.46   4.81
## 20   Control   P2    M    4  81.78 136.66
## 21   Control   P2    L    1  32.76  30.70
## 22   Control   P2    L    2  62.35 123.78
## 23   Control   P2    L    3  90.22 113.87
## 24   Control   P2    L    4 114.03  76.52
## 25 Exclusion   P3    H    1  21.86  23.58
## 26 Exclusion   P3    H    2  39.83  28.03
## 27 Exclusion   P3    H    3  59.53  21.32
## 28 Exclusion   P3    H    4  75.59  90.76
## 29 Exclusion   P3    M    1  38.57  30.63
## 30 Exclusion   P3    M    2  81.25  83.61
## 31 Exclusion   P3    M    3 124.08 124.09
## 32 Exclusion   P3    M    4 159.69 112.65
## 33 Exclusion   P3    L    1  61.16  39.53
## 34 Exclusion   P3    L    2 119.84 110.27
## 35 Exclusion   P3    L    3 175.87 286.33
## 36 Exclusion   P3    L    4 238.76  54.23
## 37 Exclusion   P4    H    1  18.82  28.60
## 38 Exclusion   P4    H    2  39.82  39.07
## 39 Exclusion   P4    H    3  63.30  93.43
## 40 Exclusion   P4    H    4  82.29  60.15
## 41 Exclusion   P4    M    1  39.51  45.90
## 42 Exclusion   P4    M    2  79.24  88.04
## 43 Exclusion   P4    M    3 122.09  84.19
## 44 Exclusion   P4    M    4 161.67 256.34
## 45 Exclusion   P4    L    1  57.93  85.24
## 46 Exclusion   P4    L    2 117.88 167.90
## 47 Exclusion   P4    L    3 181.09 314.49
## 48 Exclusion   P4    L    4 242.31 304.70

Adding columns




Mutate

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94
dat.1 %>% mutate(Sum = Resp1 + Resp2)
##    Treatment Plot Dose Time  Resp1  Resp2    Sum
## 1    Control   P1    H    1   8.12   3.06  11.18
## 2    Control   P1    H    2  20.55  25.94  46.49
## 3    Control   P1    H    3  27.49  29.85  57.34
## 4    Control   P1    H    4  44.79  25.39  70.18
## 5    Control   P1    M    1  20.99  20.31  41.30
## 6    Control   P1    M    2  37.54  17.62  55.16
## 7    Control   P1    M    3  61.46  98.44 159.90
## 8    Control   P1    M    4  82.21 160.01 242.22
## 9    Control   P1    L    1  31.73  21.22  52.95
## 10   Control   P1    L    2  59.08  37.51  96.59
## 11   Control   P1    L    3  94.54 119.22 213.76
## 12   Control   P1    L    4 121.17 116.45 237.62
## 13   Control   P2    H    1   8.14  23.93  32.07
## 14   Control   P2    H    2  13.36  28.02  41.38
## 15   Control   P2    H    3  33.37  37.17  70.54
## 16   Control   P2    H    4  39.87  38.25  78.12
## 17   Control   P2    M    1  19.95  19.73  39.68
## 18   Control   P2    M    2  42.83  40.52  83.35
## 19   Control   P2    M    3  62.46   4.81  67.27
## 20   Control   P2    M    4  81.78 136.66 218.44
## 21   Control   P2    L    1  32.76  30.70  63.46
## 22   Control   P2    L    2  62.35 123.78 186.13
## 23   Control   P2    L    3  90.22 113.87 204.09
## 24   Control   P2    L    4 114.03  76.52 190.55
## 25 Exclusion   P3    H    1  21.86  23.58  45.44
## 26 Exclusion   P3    H    2  39.83  28.03  67.86
## 27 Exclusion   P3    H    3  59.53  21.32  80.85
## 28 Exclusion   P3    H    4  75.59  90.76 166.35
## 29 Exclusion   P3    M    1  38.57  30.63  69.20
## 30 Exclusion   P3    M    2  81.25  83.61 164.86
## 31 Exclusion   P3    M    3 124.08 124.09 248.17
## 32 Exclusion   P3    M    4 159.69 112.65 272.34
## 33 Exclusion   P3    L    1  61.16  39.53 100.69
## 34 Exclusion   P3    L    2 119.84 110.27 230.11
## 35 Exclusion   P3    L    3 175.87 286.33 462.20
## 36 Exclusion   P3    L    4 238.76  54.23 292.99
## 37 Exclusion   P4    H    1  18.82  28.60  47.42
## 38 Exclusion   P4    H    2  39.82  39.07  78.89
## 39 Exclusion   P4    H    3  63.30  93.43 156.73
## 40 Exclusion   P4    H    4  82.29  60.15 142.44
## 41 Exclusion   P4    M    1  39.51  45.90  85.41
## 42 Exclusion   P4    M    2  79.24  88.04 167.28
## 43 Exclusion   P4    M    3 122.09  84.19 206.28
## 44 Exclusion   P4    M    4 161.67 256.34 418.01
## 45 Exclusion   P4    L    1  57.93  85.24 143.17
## 46 Exclusion   P4    L    2 117.88 167.90 285.78
## 47 Exclusion   P4    L    3 181.09 314.49 495.58
## 48 Exclusion   P4    L    4 242.31 304.70 547.01

Mutate

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94

Transformations

dat.1 %>% mutate(logResp1=log(Resp1))
##    Treatment Plot Dose Time  Resp1  Resp2 logResp1
## 1    Control   P1    H    1   8.12   3.06 2.094330
## 2    Control   P1    H    2  20.55  25.94 3.022861
## 3    Control   P1    H    3  27.49  29.85 3.313822
## 4    Control   P1    H    4  44.79  25.39 3.801985
## 5    Control   P1    M    1  20.99  20.31 3.044046
## 6    Control   P1    M    2  37.54  17.62 3.625407
## 7    Control   P1    M    3  61.46  98.44 4.118387
## 8    Control   P1    M    4  82.21 160.01 4.409277
## 9    Control   P1    L    1  31.73  21.22 3.457263
## 10   Control   P1    L    2  59.08  37.51 4.078892
## 11   Control   P1    L    3  94.54 119.22 4.549023
## 12   Control   P1    L    4 121.17 116.45 4.797195
## 13   Control   P2    H    1   8.14  23.93 2.096790
## 14   Control   P2    H    2  13.36  28.02 2.592265
## 15   Control   P2    H    3  33.37  37.17 3.507657
## 16   Control   P2    H    4  39.87  38.25 3.685624
## 17   Control   P2    M    1  19.95  19.73 2.993229
## 18   Control   P2    M    2  42.83  40.52 3.757239
## 19   Control   P2    M    3  62.46   4.81 4.134526
## 20   Control   P2    M    4  81.78 136.66 4.404033
## 21   Control   P2    L    1  32.76  30.70 3.489208
## 22   Control   P2    L    2  62.35 123.78 4.132764
## 23   Control   P2    L    3  90.22 113.87 4.502251
## 24   Control   P2    L    4 114.03  76.52 4.736462
## 25 Exclusion   P3    H    1  21.86  23.58 3.084658
## 26 Exclusion   P3    H    2  39.83  28.03 3.684620
## 27 Exclusion   P3    H    3  59.53  21.32 4.086480
## 28 Exclusion   P3    H    4  75.59  90.76 4.325324
## 29 Exclusion   P3    M    1  38.57  30.63 3.652475
## 30 Exclusion   P3    M    2  81.25  83.61 4.397531
## 31 Exclusion   P3    M    3 124.08 124.09 4.820927
## 32 Exclusion   P3    M    4 159.69 112.65 5.073234
## 33 Exclusion   P3    L    1  61.16  39.53 4.113493
## 34 Exclusion   P3    L    2 119.84 110.27 4.786158
## 35 Exclusion   P3    L    3 175.87 286.33 5.169745
## 36 Exclusion   P3    L    4 238.76  54.23 5.475459
## 37 Exclusion   P4    H    1  18.82  28.60 2.934920
## 38 Exclusion   P4    H    2  39.82  39.07 3.684369
## 39 Exclusion   P4    H    3  63.30  93.43 4.147885
## 40 Exclusion   P4    H    4  82.29  60.15 4.410250
## 41 Exclusion   P4    M    1  39.51  45.90 3.676554
## 42 Exclusion   P4    M    2  79.24  88.04 4.372481
## 43 Exclusion   P4    M    3 122.09  84.19 4.804758
## 44 Exclusion   P4    M    4 161.67 256.34 5.085557
## 45 Exclusion   P4    L    1  57.93  85.24 4.059235
## 46 Exclusion   P4    L    2 117.88 167.90 4.769667
## 47 Exclusion   P4    L    3 181.09 314.49 5.198994
## 48 Exclusion   P4    L    4 242.31 304.70 5.490218

Mutate

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94

Centering

dat.1 %>% mutate(MeanResp1=mean(Resp1), cResp1=Resp1-MeanResp1)
## OR if just want the centered variable..
#dat.1 %>% mutate(cResp1=Resp1-mean(Resp1))
##    Treatment Plot Dose Time  Resp1  Resp2 MeanResp1      cResp1
## 1    Control   P1    H    1   8.12   3.06  75.26604 -67.1460417
## 2    Control   P1    H    2  20.55  25.94  75.26604 -54.7160417
## 3    Control   P1    H    3  27.49  29.85  75.26604 -47.7760417
## 4    Control   P1    H    4  44.79  25.39  75.26604 -30.4760417
## 5    Control   P1    M    1  20.99  20.31  75.26604 -54.2760417
## 6    Control   P1    M    2  37.54  17.62  75.26604 -37.7260417
## 7    Control   P1    M    3  61.46  98.44  75.26604 -13.8060417
## 8    Control   P1    M    4  82.21 160.01  75.26604   6.9439583
## 9    Control   P1    L    1  31.73  21.22  75.26604 -43.5360417
## 10   Control   P1    L    2  59.08  37.51  75.26604 -16.1860417
## 11   Control   P1    L    3  94.54 119.22  75.26604  19.2739583
## 12   Control   P1    L    4 121.17 116.45  75.26604  45.9039583
## 13   Control   P2    H    1   8.14  23.93  75.26604 -67.1260417
## 14   Control   P2    H    2  13.36  28.02  75.26604 -61.9060417
## 15   Control   P2    H    3  33.37  37.17  75.26604 -41.8960417
## 16   Control   P2    H    4  39.87  38.25  75.26604 -35.3960417
## 17   Control   P2    M    1  19.95  19.73  75.26604 -55.3160417
## 18   Control   P2    M    2  42.83  40.52  75.26604 -32.4360417
## 19   Control   P2    M    3  62.46   4.81  75.26604 -12.8060417
## 20   Control   P2    M    4  81.78 136.66  75.26604   6.5139583
## 21   Control   P2    L    1  32.76  30.70  75.26604 -42.5060417
## 22   Control   P2    L    2  62.35 123.78  75.26604 -12.9160417
## 23   Control   P2    L    3  90.22 113.87  75.26604  14.9539583
## 24   Control   P2    L    4 114.03  76.52  75.26604  38.7639583
## 25 Exclusion   P3    H    1  21.86  23.58  75.26604 -53.4060417
## 26 Exclusion   P3    H    2  39.83  28.03  75.26604 -35.4360417
## 27 Exclusion   P3    H    3  59.53  21.32  75.26604 -15.7360417
## 28 Exclusion   P3    H    4  75.59  90.76  75.26604   0.3239583
## 29 Exclusion   P3    M    1  38.57  30.63  75.26604 -36.6960417
## 30 Exclusion   P3    M    2  81.25  83.61  75.26604   5.9839583
## 31 Exclusion   P3    M    3 124.08 124.09  75.26604  48.8139583
## 32 Exclusion   P3    M    4 159.69 112.65  75.26604  84.4239583
## 33 Exclusion   P3    L    1  61.16  39.53  75.26604 -14.1060417
## 34 Exclusion   P3    L    2 119.84 110.27  75.26604  44.5739583
## 35 Exclusion   P3    L    3 175.87 286.33  75.26604 100.6039583
## 36 Exclusion   P3    L    4 238.76  54.23  75.26604 163.4939583
## 37 Exclusion   P4    H    1  18.82  28.60  75.26604 -56.4460417
## 38 Exclusion   P4    H    2  39.82  39.07  75.26604 -35.4460417
## 39 Exclusion   P4    H    3  63.30  93.43  75.26604 -11.9660417
## 40 Exclusion   P4    H    4  82.29  60.15  75.26604   7.0239583
## 41 Exclusion   P4    M    1  39.51  45.90  75.26604 -35.7560417
## 42 Exclusion   P4    M    2  79.24  88.04  75.26604   3.9739583
## 43 Exclusion   P4    M    3 122.09  84.19  75.26604  46.8239583
## 44 Exclusion   P4    M    4 161.67 256.34  75.26604  86.4039583
## 45 Exclusion   P4    L    1  57.93  85.24  75.26604 -17.3360417
## 46 Exclusion   P4    L    2 117.88 167.90  75.26604  42.6139583
## 47 Exclusion   P4    L    3 181.09 314.49  75.26604 105.8239583
## 48 Exclusion   P4    L    4 242.31 304.70  75.26604 167.0439583

Mutate

dat.1 %>% head(2) %>% as_tibble
## # A tibble: 2 x 6
##   Treatment Plot  Dose   Time Resp1 Resp2
##   <fct>     <fct> <fct> <int> <dbl> <dbl>
## 1 Control   P1    H         1  8.12  3.06
## 2 Control   P1    H         2 20.6  25.9

Changing vector types (classes)

dat.1 %>% mutate(Time=factor(Time)) %>% as_tibble
## # A tibble: 48 x 6
##    Treatment Plot  Dose  Time  Resp1  Resp2
##    <fct>     <fct> <fct> <fct> <dbl>  <dbl>
##  1 Control   P1    H     1      8.12   3.06
##  2 Control   P1    H     2     20.6   25.9 
##  3 Control   P1    H     3     27.5   29.8 
##  4 Control   P1    H     4     44.8   25.4 
##  5 Control   P1    M     1     21.0   20.3 
##  6 Control   P1    M     2     37.5   17.6 
##  7 Control   P1    M     3     61.5   98.4 
##  8 Control   P1    M     4     82.2  160.  
##  9 Control   P1    L     1     31.7   21.2 
## 10 Control   P1    L     2     59.1   37.5 
## # … with 38 more rows

Mutate

dat.1 %>% head(2) %>% as_tibble
## # A tibble: 2 x 6
##   Treatment Plot  Dose   Time Resp1 Resp2
##   <fct>     <fct> <fct> <int> <dbl> <dbl>
## 1 Control   P1    H         1  8.12  3.06
## 2 Control   P1    H         2 20.6  25.9

Changing factor labels

dat.1 %>% mutate(Dose=fct_recode(Dose, High='H',  Medium='M' )) %>%
  as_tibble
## # A tibble: 48 x 6
##    Treatment Plot  Dose    Time Resp1  Resp2
##    <fct>     <fct> <fct>  <int> <dbl>  <dbl>
##  1 Control   P1    High       1  8.12   3.06
##  2 Control   P1    High       2 20.6   25.9 
##  3 Control   P1    High       3 27.5   29.8 
##  4 Control   P1    High       4 44.8   25.4 
##  5 Control   P1    Medium     1 21.0   20.3 
##  6 Control   P1    Medium     2 37.5   17.6 
##  7 Control   P1    Medium     3 61.5   98.4 
##  8 Control   P1    Medium     4 82.2  160.  
##  9 Control   P1    L          1 31.7   21.2 
## 10 Control   P1    L          2 59.1   37.5 
## # … with 38 more rows

Mutate

dat.1 %>% head(2) %>% as_tibble
## # A tibble: 2 x 6
##   Treatment Plot  Dose   Time Resp1 Resp2
##   <fct>     <fct> <fct> <int> <dbl> <dbl>
## 1 Control   P1    H         1  8.12  3.06
## 2 Control   P1    H         2 20.6  25.9

Changing factor levels

dat.1 %>% pull(Dose)
##  [1] H H H H M M M M L L L L H H H H M M M M L L L L H H H H M M M M L L L L H H H H M M M M L L L L
## Levels: H L M
dat.1 %>% mutate(Dose=fct_relevel(Dose, c('L', 'M','H'))) %>%
                    as_tibble() %>% pull(Dose)
##  [1] H H H H M M M M L L L L H H H H M M M M L L L L H H H H M M M M L L L L H H H H M M M M L L L L
## Levels: L M H

Mutate

data.1 %>% head(2) %>% as_tibble
## # A tibble: 2 x 7
##   Between Plot  Cond   Time  Temp   LAT  LONG
##   <fct>   <fct> <fct> <int> <dbl> <dbl> <dbl>
## 1 A1      P1    H         1  26.3  14.0  150.
## 2 A1      P1    M         2  27.9  12.9  145.

Changing factor levels

dat.1 %>% pull(Dose)
##  [1] H H H H M M M M L L L L H H H H M M M M L L L L H H H H M M M M L L L L H H H H M M M M L L L L
## Levels: H L M
dat.1 %>% mutate(Dose=recode_factor(Dose, 'L'='Low', 'M'='Medium')) %>%
                    as_tibble() %>% pull(Dose)
##  [1] H      H      H      H      Medium Medium Medium Medium Low    Low    Low    Low    H     
## [14] H      H      H      Medium Medium Medium Medium Low    Low    Low    Low    H      H     
## [27] H      H      Medium Medium Medium Medium Low    Low    Low    Low    H      H      H     
## [40] H      Medium Medium Medium Medium Low    Low    Low    Low   
## Levels: Low Medium H

Mutate

Window functions

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94
dat.1 %>% mutate(leadResp2=lead(Resp2), lagResp2=lag(Resp2))
##    Treatment Plot Dose Time  Resp1  Resp2 leadResp2 lagResp2
## 1    Control   P1    H    1   8.12   3.06     25.94       NA
## 2    Control   P1    H    2  20.55  25.94     29.85     3.06
## 3    Control   P1    H    3  27.49  29.85     25.39    25.94
## 4    Control   P1    H    4  44.79  25.39     20.31    29.85
## 5    Control   P1    M    1  20.99  20.31     17.62    25.39
## 6    Control   P1    M    2  37.54  17.62     98.44    20.31
## 7    Control   P1    M    3  61.46  98.44    160.01    17.62
## 8    Control   P1    M    4  82.21 160.01     21.22    98.44
## 9    Control   P1    L    1  31.73  21.22     37.51   160.01
## 10   Control   P1    L    2  59.08  37.51    119.22    21.22
## 11   Control   P1    L    3  94.54 119.22    116.45    37.51
## 12   Control   P1    L    4 121.17 116.45     23.93   119.22
## 13   Control   P2    H    1   8.14  23.93     28.02   116.45
## 14   Control   P2    H    2  13.36  28.02     37.17    23.93
## 15   Control   P2    H    3  33.37  37.17     38.25    28.02
## 16   Control   P2    H    4  39.87  38.25     19.73    37.17
## 17   Control   P2    M    1  19.95  19.73     40.52    38.25
## 18   Control   P2    M    2  42.83  40.52      4.81    19.73
## 19   Control   P2    M    3  62.46   4.81    136.66    40.52
## 20   Control   P2    M    4  81.78 136.66     30.70     4.81
## 21   Control   P2    L    1  32.76  30.70    123.78   136.66
## 22   Control   P2    L    2  62.35 123.78    113.87    30.70
## 23   Control   P2    L    3  90.22 113.87     76.52   123.78
## 24   Control   P2    L    4 114.03  76.52     23.58   113.87
## 25 Exclusion   P3    H    1  21.86  23.58     28.03    76.52
## 26 Exclusion   P3    H    2  39.83  28.03     21.32    23.58
## 27 Exclusion   P3    H    3  59.53  21.32     90.76    28.03
## 28 Exclusion   P3    H    4  75.59  90.76     30.63    21.32
## 29 Exclusion   P3    M    1  38.57  30.63     83.61    90.76
## 30 Exclusion   P3    M    2  81.25  83.61    124.09    30.63
## 31 Exclusion   P3    M    3 124.08 124.09    112.65    83.61
## 32 Exclusion   P3    M    4 159.69 112.65     39.53   124.09
## 33 Exclusion   P3    L    1  61.16  39.53    110.27   112.65
## 34 Exclusion   P3    L    2 119.84 110.27    286.33    39.53
## 35 Exclusion   P3    L    3 175.87 286.33     54.23   110.27
## 36 Exclusion   P3    L    4 238.76  54.23     28.60   286.33
## 37 Exclusion   P4    H    1  18.82  28.60     39.07    54.23
## 38 Exclusion   P4    H    2  39.82  39.07     93.43    28.60
## 39 Exclusion   P4    H    3  63.30  93.43     60.15    39.07
## 40 Exclusion   P4    H    4  82.29  60.15     45.90    93.43
## 41 Exclusion   P4    M    1  39.51  45.90     88.04    60.15
## 42 Exclusion   P4    M    2  79.24  88.04     84.19    45.90
## 43 Exclusion   P4    M    3 122.09  84.19    256.34    88.04
## 44 Exclusion   P4    M    4 161.67 256.34     85.24    84.19
## 45 Exclusion   P4    L    1  57.93  85.24    167.90   256.34
## 46 Exclusion   P4    L    2 117.88 167.90    314.49    85.24
## 47 Exclusion   P4    L    3 181.09 314.49    304.70   167.90
## 48 Exclusion   P4    L    4 242.31 304.70        NA   314.49

Mutate

Window functions

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94

Rank orders

dat.1 %>% mutate(rankTime=min_rank(Time),
                  denseRankTime=dense_rank(Time))
##    Treatment Plot Dose Time  Resp1  Resp2 rankTime denseRankTime
## 1    Control   P1    H    1   8.12   3.06        1             1
## 2    Control   P1    H    2  20.55  25.94       13             2
## 3    Control   P1    H    3  27.49  29.85       25             3
## 4    Control   P1    H    4  44.79  25.39       37             4
## 5    Control   P1    M    1  20.99  20.31        1             1
## 6    Control   P1    M    2  37.54  17.62       13             2
## 7    Control   P1    M    3  61.46  98.44       25             3
## 8    Control   P1    M    4  82.21 160.01       37             4
## 9    Control   P1    L    1  31.73  21.22        1             1
## 10   Control   P1    L    2  59.08  37.51       13             2
## 11   Control   P1    L    3  94.54 119.22       25             3
## 12   Control   P1    L    4 121.17 116.45       37             4
## 13   Control   P2    H    1   8.14  23.93        1             1
## 14   Control   P2    H    2  13.36  28.02       13             2
## 15   Control   P2    H    3  33.37  37.17       25             3
## 16   Control   P2    H    4  39.87  38.25       37             4
## 17   Control   P2    M    1  19.95  19.73        1             1
## 18   Control   P2    M    2  42.83  40.52       13             2
## 19   Control   P2    M    3  62.46   4.81       25             3
## 20   Control   P2    M    4  81.78 136.66       37             4
## 21   Control   P2    L    1  32.76  30.70        1             1
## 22   Control   P2    L    2  62.35 123.78       13             2
## 23   Control   P2    L    3  90.22 113.87       25             3
## 24   Control   P2    L    4 114.03  76.52       37             4
## 25 Exclusion   P3    H    1  21.86  23.58        1             1
## 26 Exclusion   P3    H    2  39.83  28.03       13             2
## 27 Exclusion   P3    H    3  59.53  21.32       25             3
## 28 Exclusion   P3    H    4  75.59  90.76       37             4
## 29 Exclusion   P3    M    1  38.57  30.63        1             1
## 30 Exclusion   P3    M    2  81.25  83.61       13             2
## 31 Exclusion   P3    M    3 124.08 124.09       25             3
## 32 Exclusion   P3    M    4 159.69 112.65       37             4
## 33 Exclusion   P3    L    1  61.16  39.53        1             1
## 34 Exclusion   P3    L    2 119.84 110.27       13             2
## 35 Exclusion   P3    L    3 175.87 286.33       25             3
## 36 Exclusion   P3    L    4 238.76  54.23       37             4
## 37 Exclusion   P4    H    1  18.82  28.60        1             1
## 38 Exclusion   P4    H    2  39.82  39.07       13             2
## 39 Exclusion   P4    H    3  63.30  93.43       25             3
## 40 Exclusion   P4    H    4  82.29  60.15       37             4
## 41 Exclusion   P4    M    1  39.51  45.90        1             1
## 42 Exclusion   P4    M    2  79.24  88.04       13             2
## 43 Exclusion   P4    M    3 122.09  84.19       25             3
## 44 Exclusion   P4    M    4 161.67 256.34       37             4
## 45 Exclusion   P4    L    1  57.93  85.24        1             1
## 46 Exclusion   P4    L    2 117.88 167.90       13             2
## 47 Exclusion   P4    L    3 181.09 314.49       25             3
## 48 Exclusion   P4    L    4 242.31 304.70       37             4

Mutate

Window functions

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94

Rank orders

dat.1 %>% mutate(rowresp1=row_number(Resp1), rowTime=row_number(Time),
                  rankTime=min_rank(Time))
##    Treatment Plot Dose Time  Resp1  Resp2 rowresp1 rowTime rankTime
## 1    Control   P1    H    1   8.12   3.06        1       1        1
## 2    Control   P1    H    2  20.55  25.94        6      13       13
## 3    Control   P1    H    3  27.49  29.85        9      25       25
## 4    Control   P1    H    4  44.79  25.39       20      37       37
## 5    Control   P1    M    1  20.99  20.31        7       2        1
## 6    Control   P1    M    2  37.54  17.62       13      14       13
## 7    Control   P1    M    3  61.46  98.44       25      26       25
## 8    Control   P1    M    4  82.21 160.01       33      38       37
## 9    Control   P1    L    1  31.73  21.22       10       3        1
## 10   Control   P1    L    2  59.08  37.51       22      15       13
## 11   Control   P1    L    3  94.54 119.22       36      27       25
## 12   Control   P1    L    4 121.17 116.45       40      39       37
## 13   Control   P2    H    1   8.14  23.93        2       4        1
## 14   Control   P2    H    2  13.36  28.02        3      16       13
## 15   Control   P2    H    3  33.37  37.17       12      28       25
## 16   Control   P2    H    4  39.87  38.25       18      40       37
## 17   Control   P2    M    1  19.95  19.73        5       5        1
## 18   Control   P2    M    2  42.83  40.52       19      17       13
## 19   Control   P2    M    3  62.46   4.81       27      29       25
## 20   Control   P2    M    4  81.78 136.66       32      41       37
## 21   Control   P2    L    1  32.76  30.70       11       6        1
## 22   Control   P2    L    2  62.35 123.78       26      18       13
## 23   Control   P2    L    3  90.22 113.87       35      30       25
## 24   Control   P2    L    4 114.03  76.52       37      42       37
## 25 Exclusion   P3    H    1  21.86  23.58        8       7        1
## 26 Exclusion   P3    H    2  39.83  28.03       17      19       13
## 27 Exclusion   P3    H    3  59.53  21.32       23      31       25
## 28 Exclusion   P3    H    4  75.59  90.76       29      43       37
## 29 Exclusion   P3    M    1  38.57  30.63       14       8        1
## 30 Exclusion   P3    M    2  81.25  83.61       31      20       13
## 31 Exclusion   P3    M    3 124.08 124.09       42      32       25
## 32 Exclusion   P3    M    4 159.69 112.65       43      44       37
## 33 Exclusion   P3    L    1  61.16  39.53       24       9        1
## 34 Exclusion   P3    L    2 119.84 110.27       39      21       13
## 35 Exclusion   P3    L    3 175.87 286.33       45      33       25
## 36 Exclusion   P3    L    4 238.76  54.23       47      45       37
## 37 Exclusion   P4    H    1  18.82  28.60        4      10        1
## 38 Exclusion   P4    H    2  39.82  39.07       16      22       13
## 39 Exclusion   P4    H    3  63.30  93.43       28      34       25
## 40 Exclusion   P4    H    4  82.29  60.15       34      46       37
## 41 Exclusion   P4    M    1  39.51  45.90       15      11        1
## 42 Exclusion   P4    M    2  79.24  88.04       30      23       13
## 43 Exclusion   P4    M    3 122.09  84.19       41      35       25
## 44 Exclusion   P4    M    4 161.67 256.34       44      47       37
## 45 Exclusion   P4    L    1  57.93  85.24       21      12        1
## 46 Exclusion   P4    L    2 117.88 167.90       38      24       13
## 47 Exclusion   P4    L    3 181.09 314.49       46      36       25
## 48 Exclusion   P4    L    4 242.31 304.70       48      48       37

Mutate

Window functions

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94

Rank of bins

dat.1 %>% mutate(ntile(Resp1,4))
##    Treatment Plot Dose Time  Resp1  Resp2 ntile(Resp1, 4)
## 1    Control   P1    H    1   8.12   3.06               1
## 2    Control   P1    H    2  20.55  25.94               1
## 3    Control   P1    H    3  27.49  29.85               1
## 4    Control   P1    H    4  44.79  25.39               2
## 5    Control   P1    M    1  20.99  20.31               1
## 6    Control   P1    M    2  37.54  17.62               2
## 7    Control   P1    M    3  61.46  98.44               3
## 8    Control   P1    M    4  82.21 160.01               3
## 9    Control   P1    L    1  31.73  21.22               1
## 10   Control   P1    L    2  59.08  37.51               2
## 11   Control   P1    L    3  94.54 119.22               3
## 12   Control   P1    L    4 121.17 116.45               4
## 13   Control   P2    H    1   8.14  23.93               1
## 14   Control   P2    H    2  13.36  28.02               1
## 15   Control   P2    H    3  33.37  37.17               1
## 16   Control   P2    H    4  39.87  38.25               2
## 17   Control   P2    M    1  19.95  19.73               1
## 18   Control   P2    M    2  42.83  40.52               2
## 19   Control   P2    M    3  62.46   4.81               3
## 20   Control   P2    M    4  81.78 136.66               3
## 21   Control   P2    L    1  32.76  30.70               1
## 22   Control   P2    L    2  62.35 123.78               3
## 23   Control   P2    L    3  90.22 113.87               3
## 24   Control   P2    L    4 114.03  76.52               4
## 25 Exclusion   P3    H    1  21.86  23.58               1
## 26 Exclusion   P3    H    2  39.83  28.03               2
## 27 Exclusion   P3    H    3  59.53  21.32               2
## 28 Exclusion   P3    H    4  75.59  90.76               3
## 29 Exclusion   P3    M    1  38.57  30.63               2
## 30 Exclusion   P3    M    2  81.25  83.61               3
## 31 Exclusion   P3    M    3 124.08 124.09               4
## 32 Exclusion   P3    M    4 159.69 112.65               4
## 33 Exclusion   P3    L    1  61.16  39.53               2
## 34 Exclusion   P3    L    2 119.84 110.27               4
## 35 Exclusion   P3    L    3 175.87 286.33               4
## 36 Exclusion   P3    L    4 238.76  54.23               4
## 37 Exclusion   P4    H    1  18.82  28.60               1
## 38 Exclusion   P4    H    2  39.82  39.07               2
## 39 Exclusion   P4    H    3  63.30  93.43               3
## 40 Exclusion   P4    H    4  82.29  60.15               3
## 41 Exclusion   P4    M    1  39.51  45.90               2
## 42 Exclusion   P4    M    2  79.24  88.04               3
## 43 Exclusion   P4    M    3 122.09  84.19               4
## 44 Exclusion   P4    M    4 161.67 256.34               4
## 45 Exclusion   P4    L    1  57.93  85.24               2
## 46 Exclusion   P4    L    2 117.88 167.90               4
## 47 Exclusion   P4    L    3 181.09 314.49               4
## 48 Exclusion   P4    L    4 242.31 304.70               4

Mutate

Window functions

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94

Logical bins

dat.1 %>% mutate(between(Resp1,20,40))
##    Treatment Plot Dose Time  Resp1  Resp2 between(Resp1, 20, 40)
## 1    Control   P1    H    1   8.12   3.06                  FALSE
## 2    Control   P1    H    2  20.55  25.94                   TRUE
## 3    Control   P1    H    3  27.49  29.85                   TRUE
## 4    Control   P1    H    4  44.79  25.39                  FALSE
## 5    Control   P1    M    1  20.99  20.31                   TRUE
## 6    Control   P1    M    2  37.54  17.62                   TRUE
## 7    Control   P1    M    3  61.46  98.44                  FALSE
## 8    Control   P1    M    4  82.21 160.01                  FALSE
## 9    Control   P1    L    1  31.73  21.22                   TRUE
## 10   Control   P1    L    2  59.08  37.51                  FALSE
## 11   Control   P1    L    3  94.54 119.22                  FALSE
## 12   Control   P1    L    4 121.17 116.45                  FALSE
## 13   Control   P2    H    1   8.14  23.93                  FALSE
## 14   Control   P2    H    2  13.36  28.02                  FALSE
## 15   Control   P2    H    3  33.37  37.17                   TRUE
## 16   Control   P2    H    4  39.87  38.25                   TRUE
## 17   Control   P2    M    1  19.95  19.73                  FALSE
## 18   Control   P2    M    2  42.83  40.52                  FALSE
## 19   Control   P2    M    3  62.46   4.81                  FALSE
## 20   Control   P2    M    4  81.78 136.66                  FALSE
## 21   Control   P2    L    1  32.76  30.70                   TRUE
## 22   Control   P2    L    2  62.35 123.78                  FALSE
## 23   Control   P2    L    3  90.22 113.87                  FALSE
## 24   Control   P2    L    4 114.03  76.52                  FALSE
## 25 Exclusion   P3    H    1  21.86  23.58                   TRUE
## 26 Exclusion   P3    H    2  39.83  28.03                   TRUE
## 27 Exclusion   P3    H    3  59.53  21.32                  FALSE
## 28 Exclusion   P3    H    4  75.59  90.76                  FALSE
## 29 Exclusion   P3    M    1  38.57  30.63                   TRUE
## 30 Exclusion   P3    M    2  81.25  83.61                  FALSE
## 31 Exclusion   P3    M    3 124.08 124.09                  FALSE
## 32 Exclusion   P3    M    4 159.69 112.65                  FALSE
## 33 Exclusion   P3    L    1  61.16  39.53                  FALSE
## 34 Exclusion   P3    L    2 119.84 110.27                  FALSE
## 35 Exclusion   P3    L    3 175.87 286.33                  FALSE
## 36 Exclusion   P3    L    4 238.76  54.23                  FALSE
## 37 Exclusion   P4    H    1  18.82  28.60                  FALSE
## 38 Exclusion   P4    H    2  39.82  39.07                   TRUE
## 39 Exclusion   P4    H    3  63.30  93.43                  FALSE
## 40 Exclusion   P4    H    4  82.29  60.15                  FALSE
## 41 Exclusion   P4    M    1  39.51  45.90                   TRUE
## 42 Exclusion   P4    M    2  79.24  88.04                  FALSE
## 43 Exclusion   P4    M    3 122.09  84.19                  FALSE
## 44 Exclusion   P4    M    4 161.67 256.34                  FALSE
## 45 Exclusion   P4    L    1  57.93  85.24                  FALSE
## 46 Exclusion   P4    L    2 117.88 167.90                  FALSE
## 47 Exclusion   P4    L    3 181.09 314.49                  FALSE
## 48 Exclusion   P4    L    4 242.31 304.70                  FALSE

Mutate

Window functions

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94

Categorical bins

dat.1 %>% mutate(fResp1=ifelse(Resp1<30, 'Low',
                     ifelse(between(Resp1,31,50), 'Medium', 'High')))
## OR
dat.1 %>% mutate(fResp1=case_when(Resp1<31 ~ 'Low',
                               between(Resp1, 31, 50) ~ 'Medium',
                               Resp1>50 ~ 'High'))
##    Treatment Plot Dose Time  Resp1  Resp2 fResp1
## 1    Control   P1    H    1   8.12   3.06    Low
## 2    Control   P1    H    2  20.55  25.94    Low
## 3    Control   P1    H    3  27.49  29.85    Low
## 4    Control   P1    H    4  44.79  25.39 Medium
## 5    Control   P1    M    1  20.99  20.31    Low
## 6    Control   P1    M    2  37.54  17.62 Medium
## 7    Control   P1    M    3  61.46  98.44   High
## 8    Control   P1    M    4  82.21 160.01   High
## 9    Control   P1    L    1  31.73  21.22 Medium
## 10   Control   P1    L    2  59.08  37.51   High
## 11   Control   P1    L    3  94.54 119.22   High
## 12   Control   P1    L    4 121.17 116.45   High
## 13   Control   P2    H    1   8.14  23.93    Low
## 14   Control   P2    H    2  13.36  28.02    Low
## 15   Control   P2    H    3  33.37  37.17 Medium
## 16   Control   P2    H    4  39.87  38.25 Medium
## 17   Control   P2    M    1  19.95  19.73    Low
## 18   Control   P2    M    2  42.83  40.52 Medium
## 19   Control   P2    M    3  62.46   4.81   High
## 20   Control   P2    M    4  81.78 136.66   High
## 21   Control   P2    L    1  32.76  30.70 Medium
## 22   Control   P2    L    2  62.35 123.78   High
## 23   Control   P2    L    3  90.22 113.87   High
## 24   Control   P2    L    4 114.03  76.52   High
## 25 Exclusion   P3    H    1  21.86  23.58    Low
## 26 Exclusion   P3    H    2  39.83  28.03 Medium
## 27 Exclusion   P3    H    3  59.53  21.32   High
## 28 Exclusion   P3    H    4  75.59  90.76   High
## 29 Exclusion   P3    M    1  38.57  30.63 Medium
## 30 Exclusion   P3    M    2  81.25  83.61   High
## 31 Exclusion   P3    M    3 124.08 124.09   High
## 32 Exclusion   P3    M    4 159.69 112.65   High
## 33 Exclusion   P3    L    1  61.16  39.53   High
## 34 Exclusion   P3    L    2 119.84 110.27   High
## 35 Exclusion   P3    L    3 175.87 286.33   High
## 36 Exclusion   P3    L    4 238.76  54.23   High
## 37 Exclusion   P4    H    1  18.82  28.60    Low
## 38 Exclusion   P4    H    2  39.82  39.07 Medium
## 39 Exclusion   P4    H    3  63.30  93.43   High
## 40 Exclusion   P4    H    4  82.29  60.15   High
## 41 Exclusion   P4    M    1  39.51  45.90 Medium
## 42 Exclusion   P4    M    2  79.24  88.04   High
## 43 Exclusion   P4    M    3 122.09  84.19   High
## 44 Exclusion   P4    M    4 161.67 256.34   High
## 45 Exclusion   P4    L    1  57.93  85.24   High
## 46 Exclusion   P4    L    2 117.88 167.90   High
## 47 Exclusion   P4    L    3 181.09 314.49   High
## 48 Exclusion   P4    L    4 242.31 304.70   High
##    Treatment Plot Dose Time  Resp1  Resp2 fResp1
## 1    Control   P1    H    1   8.12   3.06    Low
## 2    Control   P1    H    2  20.55  25.94    Low
## 3    Control   P1    H    3  27.49  29.85    Low
## 4    Control   P1    H    4  44.79  25.39 Medium
## 5    Control   P1    M    1  20.99  20.31    Low
## 6    Control   P1    M    2  37.54  17.62 Medium
## 7    Control   P1    M    3  61.46  98.44   High
## 8    Control   P1    M    4  82.21 160.01   High
## 9    Control   P1    L    1  31.73  21.22 Medium
## 10   Control   P1    L    2  59.08  37.51   High
## 11   Control   P1    L    3  94.54 119.22   High
## 12   Control   P1    L    4 121.17 116.45   High
## 13   Control   P2    H    1   8.14  23.93    Low
## 14   Control   P2    H    2  13.36  28.02    Low
## 15   Control   P2    H    3  33.37  37.17 Medium
## 16   Control   P2    H    4  39.87  38.25 Medium
## 17   Control   P2    M    1  19.95  19.73    Low
## 18   Control   P2    M    2  42.83  40.52 Medium
## 19   Control   P2    M    3  62.46   4.81   High
## 20   Control   P2    M    4  81.78 136.66   High
## 21   Control   P2    L    1  32.76  30.70 Medium
## 22   Control   P2    L    2  62.35 123.78   High
## 23   Control   P2    L    3  90.22 113.87   High
## 24   Control   P2    L    4 114.03  76.52   High
## 25 Exclusion   P3    H    1  21.86  23.58    Low
## 26 Exclusion   P3    H    2  39.83  28.03 Medium
## 27 Exclusion   P3    H    3  59.53  21.32   High
## 28 Exclusion   P3    H    4  75.59  90.76   High
## 29 Exclusion   P3    M    1  38.57  30.63 Medium
## 30 Exclusion   P3    M    2  81.25  83.61   High
## 31 Exclusion   P3    M    3 124.08 124.09   High
## 32 Exclusion   P3    M    4 159.69 112.65   High
## 33 Exclusion   P3    L    1  61.16  39.53   High
## 34 Exclusion   P3    L    2 119.84 110.27   High
## 35 Exclusion   P3    L    3 175.87 286.33   High
## 36 Exclusion   P3    L    4 238.76  54.23   High
## 37 Exclusion   P4    H    1  18.82  28.60    Low
## 38 Exclusion   P4    H    2  39.82  39.07 Medium
## 39 Exclusion   P4    H    3  63.30  93.43   High
## 40 Exclusion   P4    H    4  82.29  60.15   High
## 41 Exclusion   P4    M    1  39.51  45.90 Medium
## 42 Exclusion   P4    M    2  79.24  88.04   High
## 43 Exclusion   P4    M    3 122.09  84.19   High
## 44 Exclusion   P4    M    4 161.67 256.34   High
## 45 Exclusion   P4    L    1  57.93  85.24   High
## 46 Exclusion   P4    L    2 117.88 167.90   High
## 47 Exclusion   P4    L    3 181.09 314.49   High
## 48 Exclusion   P4    L    4 242.31 304.70   High

Mutate

Window functions

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94

Categorical bins

dat.1 %>% mutate(fResp1=cut(Resp1, breaks=c(0,31,50,200),
                         labels=c('Low','Medium','High')))
##    Treatment Plot Dose Time  Resp1  Resp2 fResp1
## 1    Control   P1    H    1   8.12   3.06    Low
## 2    Control   P1    H    2  20.55  25.94    Low
## 3    Control   P1    H    3  27.49  29.85    Low
## 4    Control   P1    H    4  44.79  25.39 Medium
## 5    Control   P1    M    1  20.99  20.31    Low
## 6    Control   P1    M    2  37.54  17.62 Medium
## 7    Control   P1    M    3  61.46  98.44   High
## 8    Control   P1    M    4  82.21 160.01   High
## 9    Control   P1    L    1  31.73  21.22 Medium
## 10   Control   P1    L    2  59.08  37.51   High
## 11   Control   P1    L    3  94.54 119.22   High
## 12   Control   P1    L    4 121.17 116.45   High
## 13   Control   P2    H    1   8.14  23.93    Low
## 14   Control   P2    H    2  13.36  28.02    Low
## 15   Control   P2    H    3  33.37  37.17 Medium
## 16   Control   P2    H    4  39.87  38.25 Medium
## 17   Control   P2    M    1  19.95  19.73    Low
## 18   Control   P2    M    2  42.83  40.52 Medium
## 19   Control   P2    M    3  62.46   4.81   High
## 20   Control   P2    M    4  81.78 136.66   High
## 21   Control   P2    L    1  32.76  30.70 Medium
## 22   Control   P2    L    2  62.35 123.78   High
## 23   Control   P2    L    3  90.22 113.87   High
## 24   Control   P2    L    4 114.03  76.52   High
## 25 Exclusion   P3    H    1  21.86  23.58    Low
## 26 Exclusion   P3    H    2  39.83  28.03 Medium
## 27 Exclusion   P3    H    3  59.53  21.32   High
## 28 Exclusion   P3    H    4  75.59  90.76   High
## 29 Exclusion   P3    M    1  38.57  30.63 Medium
## 30 Exclusion   P3    M    2  81.25  83.61   High
## 31 Exclusion   P3    M    3 124.08 124.09   High
## 32 Exclusion   P3    M    4 159.69 112.65   High
## 33 Exclusion   P3    L    1  61.16  39.53   High
## 34 Exclusion   P3    L    2 119.84 110.27   High
## 35 Exclusion   P3    L    3 175.87 286.33   High
## 36 Exclusion   P3    L    4 238.76  54.23   <NA>
## 37 Exclusion   P4    H    1  18.82  28.60    Low
## 38 Exclusion   P4    H    2  39.82  39.07 Medium
## 39 Exclusion   P4    H    3  63.30  93.43   High
## 40 Exclusion   P4    H    4  82.29  60.15   High
## 41 Exclusion   P4    M    1  39.51  45.90 Medium
## 42 Exclusion   P4    M    2  79.24  88.04   High
## 43 Exclusion   P4    M    3 122.09  84.19   High
## 44 Exclusion   P4    M    4 161.67 256.34   High
## 45 Exclusion   P4    L    1  57.93  85.24   High
## 46 Exclusion   P4    L    2 117.88 167.90   High
## 47 Exclusion   P4    L    3 181.09 314.49   High
## 48 Exclusion   P4    L    4 242.31 304.70   <NA>

Your turn

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94

Bin Time into Start (1 and 2) and end (3 and 4)

Your turn

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94

Bin Time into Start (1 and 2) and end (3 and 4)

Assuming even spread..

dat.1 %>% mutate(Period = cut(Time, breaks=2, labels=c('Start', 'End'))) 
##    Treatment Plot Dose Time  Resp1  Resp2 Period
## 1    Control   P1    H    1   8.12   3.06  Start
## 2    Control   P1    H    2  20.55  25.94  Start
## 3    Control   P1    H    3  27.49  29.85    End
## 4    Control   P1    H    4  44.79  25.39    End
## 5    Control   P1    M    1  20.99  20.31  Start
## 6    Control   P1    M    2  37.54  17.62  Start
## 7    Control   P1    M    3  61.46  98.44    End
## 8    Control   P1    M    4  82.21 160.01    End
## 9    Control   P1    L    1  31.73  21.22  Start
## 10   Control   P1    L    2  59.08  37.51  Start
## 11   Control   P1    L    3  94.54 119.22    End
## 12   Control   P1    L    4 121.17 116.45    End
## 13   Control   P2    H    1   8.14  23.93  Start
## 14   Control   P2    H    2  13.36  28.02  Start
## 15   Control   P2    H    3  33.37  37.17    End
## 16   Control   P2    H    4  39.87  38.25    End
## 17   Control   P2    M    1  19.95  19.73  Start
## 18   Control   P2    M    2  42.83  40.52  Start
## 19   Control   P2    M    3  62.46   4.81    End
## 20   Control   P2    M    4  81.78 136.66    End
## 21   Control   P2    L    1  32.76  30.70  Start
## 22   Control   P2    L    2  62.35 123.78  Start
## 23   Control   P2    L    3  90.22 113.87    End
## 24   Control   P2    L    4 114.03  76.52    End
## 25 Exclusion   P3    H    1  21.86  23.58  Start
## 26 Exclusion   P3    H    2  39.83  28.03  Start
## 27 Exclusion   P3    H    3  59.53  21.32    End
## 28 Exclusion   P3    H    4  75.59  90.76    End
## 29 Exclusion   P3    M    1  38.57  30.63  Start
## 30 Exclusion   P3    M    2  81.25  83.61  Start
## 31 Exclusion   P3    M    3 124.08 124.09    End
## 32 Exclusion   P3    M    4 159.69 112.65    End
## 33 Exclusion   P3    L    1  61.16  39.53  Start
## 34 Exclusion   P3    L    2 119.84 110.27  Start
## 35 Exclusion   P3    L    3 175.87 286.33    End
## 36 Exclusion   P3    L    4 238.76  54.23    End
## 37 Exclusion   P4    H    1  18.82  28.60  Start
## 38 Exclusion   P4    H    2  39.82  39.07  Start
## 39 Exclusion   P4    H    3  63.30  93.43    End
## 40 Exclusion   P4    H    4  82.29  60.15    End
## 41 Exclusion   P4    M    1  39.51  45.90  Start
## 42 Exclusion   P4    M    2  79.24  88.04  Start
## 43 Exclusion   P4    M    3 122.09  84.19    End
## 44 Exclusion   P4    M    4 161.67 256.34    End
## 45 Exclusion   P4    L    1  57.93  85.24  Start
## 46 Exclusion   P4    L    2 117.88 167.90  Start
## 47 Exclusion   P4    L    3 181.09 314.49    End
## 48 Exclusion   P4    L    4 242.31 304.70    End

Summarising (aggregating) data




Summarise

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94
dat.1 %>% summarise(MeanResp1=mean(Resp1), VarResp1=var(Resp1), N=n())
##   MeanResp1 VarResp1  N
## 1  75.26604 3200.305 48
SE <- function(x) sd(x)/sqrt(length(x))
dat.1 %>% summarise(MeanResp1=mean(Resp1), VarResp1=var(Resp1), 
          SEM=SE(Resp1))
##   MeanResp1 VarResp1      SEM
## 1  75.26604 3200.305 8.165355

Summarise

Across versions

dat.1 %>% summarise(across(c(Resp1, Resp2), list(Mean=mean, Var=var)))
##   Resp1_Mean Resp1_Var Resp2_Mean Resp2_Var
## 1   75.26604  3200.305   81.70958  5886.444
dat.1 %>% summarise(across(where(is.numeric), list(Mean=mean, Var=var)))
##   Time_Mean Time_Var Resp1_Mean Resp1_Var Resp2_Mean Resp2_Var
## 1       2.5 1.276596   75.26604  3200.305   81.70958  5886.444
dat.1 %>% summarize( across(where(is.numeric),  mean),
          across(where(is.factor),  length))
##   Time    Resp1    Resp2 Treatment Plot Dose
## 1  2.5 75.26604 81.70958        48   48   48

Summarise

Across versions

Var=c('Resp1', 'Resp2')
dat.1 %>% summarise(across(all_of(Var), list(Mean=mean, Var=var)))
##   Resp1_Mean Resp1_Var Resp2_Mean Resp2_Var
## 1   75.26604  3200.305   81.70958  5886.444

Summarise

dat.1 %>% count(Dose)
##   Dose  n
## 1    H 16
## 2    L 16
## 3    M 16
dat.1 %>% count(Dose,between(Resp1,30,50))
##   Dose between(Resp1, 30, 50)  n
## 1    H                  FALSE 11
## 2    H                   TRUE  5
## 3    L                  FALSE 14
## 4    L                   TRUE  2
## 5    M                  FALSE 12
## 6    M                   TRUE  4

Grouping (=aggregating)




Grouping

dat.1 %>% head(6)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94
## 3   Control   P1    H    3 27.49 29.85
## 4   Control   P1    H    4 44.79 25.39
## 5   Control   P1    M    1 20.99 20.31
## 6   Control   P1    M    2 37.54 17.62
dat.1 %>% group_by(Treatment,Plot) %>%
    summarise(Mean=mean(Resp1))
## # A tibble: 4 x 3
## # Groups:   Treatment [2]
##   Treatment Plot   Mean
##   <fct>     <fct> <dbl>
## 1 Control   P1     50.8
## 2 Control   P2     50.1
## 3 Exclusion P3     99.7
## 4 Exclusion P4    100.

Grouping

dat.1 %>% head(6)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94
## 3   Control   P1    H    3 27.49 29.85
## 4   Control   P1    H    4 44.79 25.39
## 5   Control   P1    M    1 20.99 20.31
## 6   Control   P1    M    2 37.54 17.62
dat.1 %>% group_by(Treatment,Plot) %>%
    summarise(Mean=mean(Resp1), Var=var(Resp1), N=n(),First=first(Resp1))
## # A tibble: 4 x 6
## # Groups:   Treatment [2]
##   Treatment Plot   Mean   Var     N First
##   <fct>     <fct> <dbl> <dbl> <int> <dbl>
## 1 Control   P1     50.8 1162.    12  8.12
## 2 Control   P2     50.1 1069.    12  8.14
## 3 Exclusion P3     99.7 4285.    12 21.9 
## 4 Exclusion P4    100.  4470.    12 18.8

Grouping

mutate vs summarise

dat.1 %>% group_by(Treatment,Plot) %>%
    summarise(Mean=mean(Resp1))
## # A tibble: 4 x 3
## # Groups:   Treatment [2]
##   Treatment Plot   Mean
##   <fct>     <fct> <dbl>
## 1 Control   P1     50.8
## 2 Control   P2     50.1
## 3 Exclusion P3     99.7
## 4 Exclusion P4    100.
dat.1 %>% group_by(Treatment,Plot) %>%
    mutate(Mean=mean(Resp1))
## # A tibble: 48 x 7
## # Groups:   Treatment, Plot [4]
##    Treatment Plot  Dose   Time Resp1  Resp2  Mean
##    <fct>     <fct> <fct> <int> <dbl>  <dbl> <dbl>
##  1 Control   P1    H         1  8.12   3.06  50.8
##  2 Control   P1    H         2 20.6   25.9   50.8
##  3 Control   P1    H         3 27.5   29.8   50.8
##  4 Control   P1    H         4 44.8   25.4   50.8
##  5 Control   P1    M         1 21.0   20.3   50.8
##  6 Control   P1    M         2 37.5   17.6   50.8
##  7 Control   P1    M         3 61.5   98.4   50.8
##  8 Control   P1    M         4 82.2  160.    50.8
##  9 Control   P1    L         1 31.7   21.2   50.8
## 10 Control   P1    L         2 59.1   37.5   50.8
## # … with 38 more rows

Grouping

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94
dat.1 %>% group_by(Treatment,Plot) %>%
  mutate(Mean=mean(Resp1), cResp1=Resp1-Mean)
## # A tibble: 48 x 8
## # Groups:   Treatment, Plot [4]
##    Treatment Plot  Dose   Time Resp1  Resp2  Mean cResp1
##    <fct>     <fct> <fct> <int> <dbl>  <dbl> <dbl>  <dbl>
##  1 Control   P1    H         1  8.12   3.06  50.8 -42.7 
##  2 Control   P1    H         2 20.6   25.9   50.8 -30.3 
##  3 Control   P1    H         3 27.5   29.8   50.8 -23.3 
##  4 Control   P1    H         4 44.8   25.4   50.8  -6.02
##  5 Control   P1    M         1 21.0   20.3   50.8 -29.8 
##  6 Control   P1    M         2 37.5   17.6   50.8 -13.3 
##  7 Control   P1    M         3 61.5   98.4   50.8  10.7 
##  8 Control   P1    M         4 82.2  160.    50.8  31.4 
##  9 Control   P1    L         1 31.7   21.2   50.8 -19.1 
## 10 Control   P1    L         2 59.1   37.5   50.8   8.27
## # … with 38 more rows

Grouping

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94
dat.1 %>% group_by(Treatment,Plot) %>%
  summarise(across(everything(), mean))
## # A tibble: 4 x 6
## # Groups:   Treatment [2]
##   Treatment Plot   Dose  Time Resp1 Resp2
##   <fct>     <fct> <dbl> <dbl> <dbl> <dbl>
## 1 Control   P1       NA   2.5  50.8  56.3
## 2 Control   P2       NA   2.5  50.1  56.2
## 3 Exclusion P3       NA   2.5  99.7  83.8
## 4 Exclusion P4       NA   2.5 100.  131.

Grouping

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94
dat.1 %>% select(-Dose,-Time) %>% group_by(Treatment,Plot) %>%
    summarise_all(list(mean))
## # A tibble: 4 x 4
## # Groups:   Treatment [2]
##   Treatment Plot  Resp1 Resp2
##   <fct>     <fct> <dbl> <dbl>
## 1 Control   P1     50.8  56.3
## 2 Control   P2     50.1  56.2
## 3 Exclusion P3     99.7  83.8
## 4 Exclusion P4    100.  131.

Grouping

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94
dat.1 %>% group_by(Treatment,Plot) %>%
    summarise(across(c(Time,Resp1,Resp2), list(Mean=mean, SE=SE)))
## # A tibble: 4 x 8
## # Groups:   Treatment [2]
##   Treatment Plot  Time_Mean Time_SE Resp1_Mean Resp1_SE Resp2_Mean
##   <fct>     <fct>     <dbl>   <dbl>      <dbl>    <dbl>      <dbl>
## 1 Control   P1          2.5   0.337       50.8     9.84       56.3
## 2 Control   P2          2.5   0.337       50.1     9.44       56.2
## 3 Exclusion P3          2.5   0.337       99.7    18.9        83.8
## 4 Exclusion P4          2.5   0.337      100.     19.3       131. 
## # … with 1 more variable: Resp2_SE <dbl>

Your turn

Calculate for each year, the mean abundance of Pocillopora damicornis

tikus[1:10,c(1:3,76:77)]
##     Psammocora contigua Psammocora digitata Pocillopora damicornis time rep
## V1                    0                   0                     79   81   1
## V2                    0                   0                     51   81   2
## V3                    0                   0                     42   81   3
## V4                    0                   0                     15   81   4
## V5                    0                   0                      9   81   5
## V6                    0                   0                     72   81   6
## V7                    0                   0                      0   81   7
## V8                    0                   0                     16   81   8
## V9                    0                   0                      0   81   9
## V10                   0                   0                     16   81  10

NOTE to operate on columns whose names contain special characters (including spaces), you must use `` (backticks).

tikus %>% arrange(`Pocillopora damicornis`)

Your turn

Calculate for each year, the mean abundance of Pocillopora damicornis

tikus %>% group_by(time) %>%
    summarise(MeanAbundance=mean(`Pocillopora damicornis`))
## # A tibble: 6 x 2
##   time  MeanAbundance
## * <fct>         <dbl>
## 1 81             30  
## 2 83              0  
## 3 84              0  
## 4 85              0  
## 5 87              1.8
## 6 88              4

Your turn

Calculate for each year, the number of samples as well as the mean and variance of ozone

nasa = as.data.frame(nasa)
head(nasa)
##        lat   long month year cloudhigh cloudlow cloudmid ozone pressure surftemp temperature
## 1 36.20000 -113.8     1 1995      26.0      7.5     34.5   304      835    272.7       272.1
## 2 33.70435 -113.8     1 1995      20.0     11.5     32.5   304      940    279.5       282.2
## 3 31.20870 -113.8     1 1995      16.0     16.5     26.0   298      960    284.7       285.2
## 4 28.71304 -113.8     1 1995      13.0     20.5     14.5   276      990    289.3       290.7
## 5 26.21739 -113.8     1 1995       7.5     26.0     10.5   274     1000    292.2       292.7
## 6 23.72174 -113.8     1 1995       8.0     30.0      9.5   264     1000    294.1       293.6

Your turn

Calculate for each year, the number of samples as well as the mean and variance of ozone

nasa %>% group_by(year) %>%
    summarise(N=n(),Mean=mean(ozone), Var=var(ozone))
## # A tibble: 6 x 4
##    year     N  Mean   Var
## * <int> <int> <dbl> <dbl>
## 1  1995  6912  264.  258.
## 2  1996  6912  267.  326.
## 3  1997  6912  266.  327.
## 4  1998  6912  267.  507.
## 5  1999  6912  270.  368.
## 6  2000  6912  269.  353.

Subset columns




Selecting columns (select)

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94
dat.1 %>% select(Treatment,Plot,Dose,Time,Resp1)
##    Treatment Plot Dose Time  Resp1
## 1    Control   P1    H    1   8.12
## 2    Control   P1    H    2  20.55
## 3    Control   P1    H    3  27.49
## 4    Control   P1    H    4  44.79
## 5    Control   P1    M    1  20.99
## 6    Control   P1    M    2  37.54
## 7    Control   P1    M    3  61.46
## 8    Control   P1    M    4  82.21
## 9    Control   P1    L    1  31.73
## 10   Control   P1    L    2  59.08
## 11   Control   P1    L    3  94.54
## 12   Control   P1    L    4 121.17
## 13   Control   P2    H    1   8.14
## 14   Control   P2    H    2  13.36
## 15   Control   P2    H    3  33.37
## 16   Control   P2    H    4  39.87
## 17   Control   P2    M    1  19.95
## 18   Control   P2    M    2  42.83
## 19   Control   P2    M    3  62.46
## 20   Control   P2    M    4  81.78
## 21   Control   P2    L    1  32.76
## 22   Control   P2    L    2  62.35
## 23   Control   P2    L    3  90.22
## 24   Control   P2    L    4 114.03
## 25 Exclusion   P3    H    1  21.86
## 26 Exclusion   P3    H    2  39.83
## 27 Exclusion   P3    H    3  59.53
## 28 Exclusion   P3    H    4  75.59
## 29 Exclusion   P3    M    1  38.57
## 30 Exclusion   P3    M    2  81.25
## 31 Exclusion   P3    M    3 124.08
## 32 Exclusion   P3    M    4 159.69
## 33 Exclusion   P3    L    1  61.16
## 34 Exclusion   P3    L    2 119.84
## 35 Exclusion   P3    L    3 175.87
## 36 Exclusion   P3    L    4 238.76
## 37 Exclusion   P4    H    1  18.82
## 38 Exclusion   P4    H    2  39.82
## 39 Exclusion   P4    H    3  63.30
## 40 Exclusion   P4    H    4  82.29
## 41 Exclusion   P4    M    1  39.51
## 42 Exclusion   P4    M    2  79.24
## 43 Exclusion   P4    M    3 122.09
## 44 Exclusion   P4    M    4 161.67
## 45 Exclusion   P4    L    1  57.93
## 46 Exclusion   P4    L    2 117.88
## 47 Exclusion   P4    L    3 181.09
## 48 Exclusion   P4    L    4 242.31

Selecting columns (select)

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94
dat.1 %>% select(-Resp2)
##    Treatment Plot Dose Time  Resp1
## 1    Control   P1    H    1   8.12
## 2    Control   P1    H    2  20.55
## 3    Control   P1    H    3  27.49
## 4    Control   P1    H    4  44.79
## 5    Control   P1    M    1  20.99
## 6    Control   P1    M    2  37.54
## 7    Control   P1    M    3  61.46
## 8    Control   P1    M    4  82.21
## 9    Control   P1    L    1  31.73
## 10   Control   P1    L    2  59.08
## 11   Control   P1    L    3  94.54
## 12   Control   P1    L    4 121.17
## 13   Control   P2    H    1   8.14
## 14   Control   P2    H    2  13.36
## 15   Control   P2    H    3  33.37
## 16   Control   P2    H    4  39.87
## 17   Control   P2    M    1  19.95
## 18   Control   P2    M    2  42.83
## 19   Control   P2    M    3  62.46
## 20   Control   P2    M    4  81.78
## 21   Control   P2    L    1  32.76
## 22   Control   P2    L    2  62.35
## 23   Control   P2    L    3  90.22
## 24   Control   P2    L    4 114.03
## 25 Exclusion   P3    H    1  21.86
## 26 Exclusion   P3    H    2  39.83
## 27 Exclusion   P3    H    3  59.53
## 28 Exclusion   P3    H    4  75.59
## 29 Exclusion   P3    M    1  38.57
## 30 Exclusion   P3    M    2  81.25
## 31 Exclusion   P3    M    3 124.08
## 32 Exclusion   P3    M    4 159.69
## 33 Exclusion   P3    L    1  61.16
## 34 Exclusion   P3    L    2 119.84
## 35 Exclusion   P3    L    3 175.87
## 36 Exclusion   P3    L    4 238.76
## 37 Exclusion   P4    H    1  18.82
## 38 Exclusion   P4    H    2  39.82
## 39 Exclusion   P4    H    3  63.30
## 40 Exclusion   P4    H    4  82.29
## 41 Exclusion   P4    M    1  39.51
## 42 Exclusion   P4    M    2  79.24
## 43 Exclusion   P4    M    3 122.09
## 44 Exclusion   P4    M    4 161.67
## 45 Exclusion   P4    L    1  57.93
## 46 Exclusion   P4    L    2 117.88
## 47 Exclusion   P4    L    3 181.09
## 48 Exclusion   P4    L    4 242.31

Selecting columns (select)

helper functions

  • contains()
  • ends_with()
  • starts_with()
  • matches()
  • everything()
  • across()

must evaluate to TRUE/FALSE

Selecting columns (select)

helper functions

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94
dat.1 %>% select(contains('R'))
##    Treatment  Resp1  Resp2
## 1    Control   8.12   3.06
## 2    Control  20.55  25.94
## 3    Control  27.49  29.85
## 4    Control  44.79  25.39
## 5    Control  20.99  20.31
## 6    Control  37.54  17.62
## 7    Control  61.46  98.44
## 8    Control  82.21 160.01
## 9    Control  31.73  21.22
## 10   Control  59.08  37.51
## 11   Control  94.54 119.22
## 12   Control 121.17 116.45
## 13   Control   8.14  23.93
## 14   Control  13.36  28.02
## 15   Control  33.37  37.17
## 16   Control  39.87  38.25
## 17   Control  19.95  19.73
## 18   Control  42.83  40.52
## 19   Control  62.46   4.81
## 20   Control  81.78 136.66
## 21   Control  32.76  30.70
## 22   Control  62.35 123.78
## 23   Control  90.22 113.87
## 24   Control 114.03  76.52
## 25 Exclusion  21.86  23.58
## 26 Exclusion  39.83  28.03
## 27 Exclusion  59.53  21.32
## 28 Exclusion  75.59  90.76
## 29 Exclusion  38.57  30.63
## 30 Exclusion  81.25  83.61
## 31 Exclusion 124.08 124.09
## 32 Exclusion 159.69 112.65
## 33 Exclusion  61.16  39.53
## 34 Exclusion 119.84 110.27
## 35 Exclusion 175.87 286.33
## 36 Exclusion 238.76  54.23
## 37 Exclusion  18.82  28.60
## 38 Exclusion  39.82  39.07
## 39 Exclusion  63.30  93.43
## 40 Exclusion  82.29  60.15
## 41 Exclusion  39.51  45.90
## 42 Exclusion  79.24  88.04
## 43 Exclusion 122.09  84.19
## 44 Exclusion 161.67 256.34
## 45 Exclusion  57.93  85.24
## 46 Exclusion 117.88 167.90
## 47 Exclusion 181.09 314.49
## 48 Exclusion 242.31 304.70

Selecting columns (select)

helper functions

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94
dat.1 %>% select(starts_with('R'))
##     Resp1  Resp2
## 1    8.12   3.06
## 2   20.55  25.94
## 3   27.49  29.85
## 4   44.79  25.39
## 5   20.99  20.31
## 6   37.54  17.62
## 7   61.46  98.44
## 8   82.21 160.01
## 9   31.73  21.22
## 10  59.08  37.51
## 11  94.54 119.22
## 12 121.17 116.45
## 13   8.14  23.93
## 14  13.36  28.02
## 15  33.37  37.17
## 16  39.87  38.25
## 17  19.95  19.73
## 18  42.83  40.52
## 19  62.46   4.81
## 20  81.78 136.66
## 21  32.76  30.70
## 22  62.35 123.78
## 23  90.22 113.87
## 24 114.03  76.52
## 25  21.86  23.58
## 26  39.83  28.03
## 27  59.53  21.32
## 28  75.59  90.76
## 29  38.57  30.63
## 30  81.25  83.61
## 31 124.08 124.09
## 32 159.69 112.65
## 33  61.16  39.53
## 34 119.84 110.27
## 35 175.87 286.33
## 36 238.76  54.23
## 37  18.82  28.60
## 38  39.82  39.07
## 39  63.30  93.43
## 40  82.29  60.15
## 41  39.51  45.90
## 42  79.24  88.04
## 43 122.09  84.19
## 44 161.67 256.34
## 45  57.93  85.24
## 46 117.88 167.90
## 47 181.09 314.49
## 48 242.31 304.70

Selecting columns (select)

helper functions

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94
dat.1 %>% select(ends_with('e'))
##    Dose Time
## 1     H    1
## 2     H    2
## 3     H    3
## 4     H    4
## 5     M    1
## 6     M    2
## 7     M    3
## 8     M    4
## 9     L    1
## 10    L    2
## 11    L    3
## 12    L    4
## 13    H    1
## 14    H    2
## 15    H    3
## 16    H    4
## 17    M    1
## 18    M    2
## 19    M    3
## 20    M    4
## 21    L    1
## 22    L    2
## 23    L    3
## 24    L    4
## 25    H    1
## 26    H    2
## 27    H    3
## 28    H    4
## 29    M    1
## 30    M    2
## 31    M    3
## 32    M    4
## 33    L    1
## 34    L    2
## 35    L    3
## 36    L    4
## 37    H    1
## 38    H    2
## 39    H    3
## 40    H    4
## 41    M    1
## 42    M    2
## 43    M    3
## 44    M    4
## 45    L    1
## 46    L    2
## 47    L    3
## 48    L    4

Selecting columns (select)

helper functions

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94
dat.1 %>% select(matches('^.{4}$'))
##    Plot Dose Time
## 1    P1    H    1
## 2    P1    H    2
## 3    P1    H    3
## 4    P1    H    4
## 5    P1    M    1
## 6    P1    M    2
## 7    P1    M    3
## 8    P1    M    4
## 9    P1    L    1
## 10   P1    L    2
## 11   P1    L    3
## 12   P1    L    4
## 13   P2    H    1
## 14   P2    H    2
## 15   P2    H    3
## 16   P2    H    4
## 17   P2    M    1
## 18   P2    M    2
## 19   P2    M    3
## 20   P2    M    4
## 21   P2    L    1
## 22   P2    L    2
## 23   P2    L    3
## 24   P2    L    4
## 25   P3    H    1
## 26   P3    H    2
## 27   P3    H    3
## 28   P3    H    4
## 29   P3    M    1
## 30   P3    M    2
## 31   P3    M    3
## 32   P3    M    4
## 33   P3    L    1
## 34   P3    L    2
## 35   P3    L    3
## 36   P3    L    4
## 37   P4    H    1
## 38   P4    H    2
## 39   P4    H    3
## 40   P4    H    4
## 41   P4    M    1
## 42   P4    M    2
## 43   P4    M    3
## 44   P4    M    4
## 45   P4    L    1
## 46   P4    L    2
## 47   P4    L    3
## 48   P4    L    4

Regular expressions (regexp)

Selecting columns (select)

helper functions

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94
dat.1 %>% select(Treatment:Time)
##    Treatment Plot Dose Time
## 1    Control   P1    H    1
## 2    Control   P1    H    2
## 3    Control   P1    H    3
## 4    Control   P1    H    4
## 5    Control   P1    M    1
## 6    Control   P1    M    2
## 7    Control   P1    M    3
## 8    Control   P1    M    4
## 9    Control   P1    L    1
## 10   Control   P1    L    2
## 11   Control   P1    L    3
## 12   Control   P1    L    4
## 13   Control   P2    H    1
## 14   Control   P2    H    2
## 15   Control   P2    H    3
## 16   Control   P2    H    4
## 17   Control   P2    M    1
## 18   Control   P2    M    2
## 19   Control   P2    M    3
## 20   Control   P2    M    4
## 21   Control   P2    L    1
## 22   Control   P2    L    2
## 23   Control   P2    L    3
## 24   Control   P2    L    4
## 25 Exclusion   P3    H    1
## 26 Exclusion   P3    H    2
## 27 Exclusion   P3    H    3
## 28 Exclusion   P3    H    4
## 29 Exclusion   P3    M    1
## 30 Exclusion   P3    M    2
## 31 Exclusion   P3    M    3
## 32 Exclusion   P3    M    4
## 33 Exclusion   P3    L    1
## 34 Exclusion   P3    L    2
## 35 Exclusion   P3    L    3
## 36 Exclusion   P3    L    4
## 37 Exclusion   P4    H    1
## 38 Exclusion   P4    H    2
## 39 Exclusion   P4    H    3
## 40 Exclusion   P4    H    4
## 41 Exclusion   P4    M    1
## 42 Exclusion   P4    M    2
## 43 Exclusion   P4    M    3
## 44 Exclusion   P4    M    4
## 45 Exclusion   P4    L    1
## 46 Exclusion   P4    L    2
## 47 Exclusion   P4    L    3
## 48 Exclusion   P4    L    4

Your turn

nasa %>% head()
##        lat   long month year cloudhigh cloudlow cloudmid ozone pressure surftemp temperature
## 1 36.20000 -113.8     1 1995      26.0      7.5     34.5   304      835    272.7       272.1
## 2 33.70435 -113.8     1 1995      20.0     11.5     32.5   304      940    279.5       282.2
## 3 31.20870 -113.8     1 1995      16.0     16.5     26.0   298      960    284.7       285.2
## 4 28.71304 -113.8     1 1995      13.0     20.5     14.5   276      990    289.3       290.7
## 5 26.21739 -113.8     1 1995       7.5     26.0     10.5   274     1000    292.2       292.7
## 6 23.72174 -113.8     1 1995       8.0     30.0      9.5   264     1000    294.1       293.6

Select lat, long, and cloud.. columns

Your turn

nasa %>% head()
##        lat   long month year cloudhigh cloudlow cloudmid ozone pressure surftemp temperature
## 1 36.20000 -113.8     1 1995      26.0      7.5     34.5   304      835    272.7       272.1
## 2 33.70435 -113.8     1 1995      20.0     11.5     32.5   304      940    279.5       282.2
## 3 31.20870 -113.8     1 1995      16.0     16.5     26.0   298      960    284.7       285.2
## 4 28.71304 -113.8     1 1995      13.0     20.5     14.5   276      990    289.3       290.7
## 5 26.21739 -113.8     1 1995       7.5     26.0     10.5   274     1000    292.2       292.7
## 6 23.72174 -113.8     1 1995       8.0     30.0      9.5   264     1000    294.1       293.6
nasa %>% select(lat, long, starts_with("cloud")) %>% head
##        lat   long cloudhigh cloudlow cloudmid
## 1 36.20000 -113.8      26.0      7.5     34.5
## 2 33.70435 -113.8      20.0     11.5     32.5
## 3 31.20870 -113.8      16.0     16.5     26.0
## 4 28.71304 -113.8      13.0     20.5     14.5
## 5 26.21739 -113.8       7.5     26.0     10.5
## 6 23.72174 -113.8       8.0     30.0      9.5

Your turn

tikus[1:10,c(1:3,76:77)]
##     Psammocora contigua Psammocora digitata Pocillopora damicornis time rep
## V1                    0                   0                     79   81   1
## V2                    0                   0                     51   81   2
## V3                    0                   0                     42   81   3
## V4                    0                   0                     15   81   4
## V5                    0                   0                      9   81   5
## V6                    0                   0                     72   81   6
## V7                    0                   0                      0   81   7
## V8                    0                   0                     16   81   8
## V9                    0                   0                      0   81   9
## V10                   0                   0                     16   81  10

Select rep, time and only Species that DONT contain pora

Your turn

Select rep, time and only Species that DONT contain pora

tikas %>% dplyr::select(-contains('pora'))
## OR if we wanted to alter the order...
tikas %>% dplyr::select(rep, time, everything(),-contains('pora'))

Select awkward names

dplyr::select(tikus, `Pocillopora damicornis`)
##     Pocillopora damicornis
## V1                      79
## V2                      51
## V3                      42
## V4                      15
## V5                       9
## V6                      72
## V7                       0
## V8                      16
## V9                       0
## V10                     16
## V11                      0
## V12                      0
## V13                      0
## V14                      0
## V15                      0
## V16                      0
## V17                      0
## V18                      0
## V19                      0
## V20                      0
## V21                      0
## V22                      0
## V23                      0
## V24                      0
## V25                      0
## V26                      0
## V27                      0
## V28                      0
## V29                      0
## V30                      0
## V31                      0
## V32                      0
## V33                      0
## V34                      0
## V35                      0
## V36                      0
## V37                      0
## V38                      0
## V39                      0
## V40                      0
## V41                     18
## V42                      0
## V43                      0
## V44                      0
## V45                      0
## V46                      0
## V47                      0
## V48                      0
## V49                      0
## V50                      0
## V51                      0
## V52                      0
## V53                      0
## V54                      0
## V55                      0
## V56                      0
## V57                     10
## V58                      0
## V59                     30
## V60                      0

Selecting a single variable

dat.1 %>% pull(Resp1)
##  [1]   8.12  20.55  27.49  44.79  20.99  37.54  61.46  82.21  31.73  59.08  94.54 121.17
## [13]   8.14  13.36  33.37  39.87  19.95  42.83  62.46  81.78  32.76  62.35  90.22 114.03
## [25]  21.86  39.83  59.53  75.59  38.57  81.25 124.08 159.69  61.16 119.84 175.87 238.76
## [37]  18.82  39.82  63.30  82.29  39.51  79.24 122.09 161.67  57.93 117.88 181.09 242.31

Re-naming columns (vectors)

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94
dat.1 %>% rename(Exposure=Treatment, Richness=Resp1)
##     Exposure Plot Dose Time Richness  Resp2
## 1    Control   P1    H    1     8.12   3.06
## 2    Control   P1    H    2    20.55  25.94
## 3    Control   P1    H    3    27.49  29.85
## 4    Control   P1    H    4    44.79  25.39
## 5    Control   P1    M    1    20.99  20.31
## 6    Control   P1    M    2    37.54  17.62
## 7    Control   P1    M    3    61.46  98.44
## 8    Control   P1    M    4    82.21 160.01
## 9    Control   P1    L    1    31.73  21.22
## 10   Control   P1    L    2    59.08  37.51
## 11   Control   P1    L    3    94.54 119.22
## 12   Control   P1    L    4   121.17 116.45
## 13   Control   P2    H    1     8.14  23.93
## 14   Control   P2    H    2    13.36  28.02
## 15   Control   P2    H    3    33.37  37.17
## 16   Control   P2    H    4    39.87  38.25
## 17   Control   P2    M    1    19.95  19.73
## 18   Control   P2    M    2    42.83  40.52
## 19   Control   P2    M    3    62.46   4.81
## 20   Control   P2    M    4    81.78 136.66
## 21   Control   P2    L    1    32.76  30.70
## 22   Control   P2    L    2    62.35 123.78
## 23   Control   P2    L    3    90.22 113.87
## 24   Control   P2    L    4   114.03  76.52
## 25 Exclusion   P3    H    1    21.86  23.58
## 26 Exclusion   P3    H    2    39.83  28.03
## 27 Exclusion   P3    H    3    59.53  21.32
## 28 Exclusion   P3    H    4    75.59  90.76
## 29 Exclusion   P3    M    1    38.57  30.63
## 30 Exclusion   P3    M    2    81.25  83.61
## 31 Exclusion   P3    M    3   124.08 124.09
## 32 Exclusion   P3    M    4   159.69 112.65
## 33 Exclusion   P3    L    1    61.16  39.53
## 34 Exclusion   P3    L    2   119.84 110.27
## 35 Exclusion   P3    L    3   175.87 286.33
## 36 Exclusion   P3    L    4   238.76  54.23
## 37 Exclusion   P4    H    1    18.82  28.60
## 38 Exclusion   P4    H    2    39.82  39.07
## 39 Exclusion   P4    H    3    63.30  93.43
## 40 Exclusion   P4    H    4    82.29  60.15
## 41 Exclusion   P4    M    1    39.51  45.90
## 42 Exclusion   P4    M    2    79.24  88.04
## 43 Exclusion   P4    M    3   122.09  84.19
## 44 Exclusion   P4    M    4   161.67 256.34
## 45 Exclusion   P4    L    1    57.93  85.24
## 46 Exclusion   P4    L    2   117.88 167.90
## 47 Exclusion   P4    L    3   181.09 314.49
## 48 Exclusion   P4    L    4   242.31 304.70

Filtering




Filtering

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94
dat.1 %>% filter(Dose=='H')
##    Treatment Plot Dose Time Resp1 Resp2
## 1    Control   P1    H    1  8.12  3.06
## 2    Control   P1    H    2 20.55 25.94
## 3    Control   P1    H    3 27.49 29.85
## 4    Control   P1    H    4 44.79 25.39
## 5    Control   P2    H    1  8.14 23.93
## 6    Control   P2    H    2 13.36 28.02
## 7    Control   P2    H    3 33.37 37.17
## 8    Control   P2    H    4 39.87 38.25
## 9  Exclusion   P3    H    1 21.86 23.58
## 10 Exclusion   P3    H    2 39.83 28.03
## 11 Exclusion   P3    H    3 59.53 21.32
## 12 Exclusion   P3    H    4 75.59 90.76
## 13 Exclusion   P4    H    1 18.82 28.60
## 14 Exclusion   P4    H    2 39.82 39.07
## 15 Exclusion   P4    H    3 63.30 93.43
## 16 Exclusion   P4    H    4 82.29 60.15
dat.1 %>% filter(Dose %in% c('H','M'))
##    Treatment Plot Dose Time  Resp1  Resp2
## 1    Control   P1    H    1   8.12   3.06
## 2    Control   P1    H    2  20.55  25.94
## 3    Control   P1    H    3  27.49  29.85
## 4    Control   P1    H    4  44.79  25.39
## 5    Control   P1    M    1  20.99  20.31
## 6    Control   P1    M    2  37.54  17.62
## 7    Control   P1    M    3  61.46  98.44
## 8    Control   P1    M    4  82.21 160.01
## 9    Control   P2    H    1   8.14  23.93
## 10   Control   P2    H    2  13.36  28.02
## 11   Control   P2    H    3  33.37  37.17
## 12   Control   P2    H    4  39.87  38.25
## 13   Control   P2    M    1  19.95  19.73
## 14   Control   P2    M    2  42.83  40.52
## 15   Control   P2    M    3  62.46   4.81
## 16   Control   P2    M    4  81.78 136.66
## 17 Exclusion   P3    H    1  21.86  23.58
## 18 Exclusion   P3    H    2  39.83  28.03
## 19 Exclusion   P3    H    3  59.53  21.32
## 20 Exclusion   P3    H    4  75.59  90.76
## 21 Exclusion   P3    M    1  38.57  30.63
## 22 Exclusion   P3    M    2  81.25  83.61
## 23 Exclusion   P3    M    3 124.08 124.09
## 24 Exclusion   P3    M    4 159.69 112.65
## 25 Exclusion   P4    H    1  18.82  28.60
## 26 Exclusion   P4    H    2  39.82  39.07
## 27 Exclusion   P4    H    3  63.30  93.43
## 28 Exclusion   P4    H    4  82.29  60.15
## 29 Exclusion   P4    M    1  39.51  45.90
## 30 Exclusion   P4    M    2  79.24  88.04
## 31 Exclusion   P4    M    3 122.09  84.19
## 32 Exclusion   P4    M    4 161.67 256.34

Filtering

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94
dat.1 %>% filter(Dose=='H' & Resp1<25)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94
## 3   Control   P2    H    1  8.14 23.93
## 4   Control   P2    H    2 13.36 28.02
## 5 Exclusion   P3    H    1 21.86 23.58
## 6 Exclusion   P4    H    1 18.82 28.60
dat.1 %>% filter(Dose=='H' | Resp1<25)
##    Treatment Plot Dose Time Resp1 Resp2
## 1    Control   P1    H    1  8.12  3.06
## 2    Control   P1    H    2 20.55 25.94
## 3    Control   P1    H    3 27.49 29.85
## 4    Control   P1    H    4 44.79 25.39
## 5    Control   P1    M    1 20.99 20.31
## 6    Control   P2    H    1  8.14 23.93
## 7    Control   P2    H    2 13.36 28.02
## 8    Control   P2    H    3 33.37 37.17
## 9    Control   P2    H    4 39.87 38.25
## 10   Control   P2    M    1 19.95 19.73
## 11 Exclusion   P3    H    1 21.86 23.58
## 12 Exclusion   P3    H    2 39.83 28.03
## 13 Exclusion   P3    H    3 59.53 21.32
## 14 Exclusion   P3    H    4 75.59 90.76
## 15 Exclusion   P4    H    1 18.82 28.60
## 16 Exclusion   P4    H    2 39.82 39.07
## 17 Exclusion   P4    H    3 63.30 93.43
## 18 Exclusion   P4    H    4 82.29 60.15

Your turn

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94

Keep only those rows with Resp1 less than 40 and Time greater than 1 or Dose is equal to L

Your turn

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94

Keep only those rows with Resp1 less than 40 and Time greater than 1 or Dose is equal to L

dat.1 %>% filter(Resp1<40 & (Time>1 |  Dose =="L"))
##    Treatment Plot Dose Time Resp1 Resp2
## 1    Control   P1    H    2 20.55 25.94
## 2    Control   P1    H    3 27.49 29.85
## 3    Control   P1    M    2 37.54 17.62
## 4    Control   P1    L    1 31.73 21.22
## 5    Control   P2    H    2 13.36 28.02
## 6    Control   P2    H    3 33.37 37.17
## 7    Control   P2    H    4 39.87 38.25
## 8    Control   P2    L    1 32.76 30.70
## 9  Exclusion   P3    H    2 39.83 28.03
## 10 Exclusion   P4    H    2 39.82 39.07

Your turn

glimpse(nasa)
## Rows: 41,472
## Columns: 11
## $ lat         <dbl> 36.200000, 33.704348, 31.208696, 28.713043, 26.217391, 23.721739, 21…
## $ long        <dbl> -113.8000, -113.8000, -113.8000, -113.8000, -113.8000, -113.8000, -1…
## $ month       <int> 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,…
## $ year        <int> 1995, 1995, 1995, 1995, 1995, 1995, 1995, 1995, 1995, 1995, 1995, 19…
## $ cloudhigh   <dbl> 26.0, 20.0, 16.0, 13.0, 7.5, 8.0, 14.5, 19.5, 22.5, 21.0, 19.0, 16.5…
## $ cloudlow    <dbl> 7.5, 11.5, 16.5, 20.5, 26.0, 30.0, 29.5, 26.5, 27.5, 26.0, 28.5, 28.…
## $ cloudmid    <dbl> 34.5, 32.5, 26.0, 14.5, 10.5, 9.5, 11.0, 17.5, 18.5, 16.5, 12.5, 13.…
## $ ozone       <dbl> 304, 304, 298, 276, 274, 264, 258, 252, 250, 250, 248, 248, 250, 248…
## $ pressure    <dbl> 835, 940, 960, 990, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, …
## $ surftemp    <dbl> 272.7, 279.5, 284.7, 289.3, 292.2, 294.1, 295.0, 298.3, 300.1, 300.1…
## $ temperature <dbl> 272.1, 282.2, 285.2, 290.7, 292.7, 293.6, 294.6, 296.9, 297.8, 298.7…

Filter to the largest ozone value for the second month of the last year

Your turn

Filter to the largest ozone value for the second month of the last year

nasa %>% filter(year==max(year) & month==2) %>% 
    arrange(-ozone) %>% head(5)
nasa %>% filter(year==max(year) & month==2) %>%
    arrange(-ozone) %>% slice(1:5)
##OR
nasa %>% filter(year==max(year) & month==2 ) %>%
    top_n(5, ozone)

Your turn

Filter to all ozone values between 320 and 325 in the first month of the last year

glimpse(nasa)
## Rows: 41,472
## Columns: 11
## $ lat         <dbl> 36.200000, 33.704348, 31.208696, 28.713043, 26.217391, 23.721739, 21…
## $ long        <dbl> -113.8000, -113.8000, -113.8000, -113.8000, -113.8000, -113.8000, -1…
## $ month       <int> 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,…
## $ year        <int> 1995, 1995, 1995, 1995, 1995, 1995, 1995, 1995, 1995, 1995, 1995, 19…
## $ cloudhigh   <dbl> 26.0, 20.0, 16.0, 13.0, 7.5, 8.0, 14.5, 19.5, 22.5, 21.0, 19.0, 16.5…
## $ cloudlow    <dbl> 7.5, 11.5, 16.5, 20.5, 26.0, 30.0, 29.5, 26.5, 27.5, 26.0, 28.5, 28.…
## $ cloudmid    <dbl> 34.5, 32.5, 26.0, 14.5, 10.5, 9.5, 11.0, 17.5, 18.5, 16.5, 12.5, 13.…
## $ ozone       <dbl> 304, 304, 298, 276, 274, 264, 258, 252, 250, 250, 248, 248, 250, 248…
## $ pressure    <dbl> 835, 940, 960, 990, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, …
## $ surftemp    <dbl> 272.7, 279.5, 284.7, 289.3, 292.2, 294.1, 295.0, 298.3, 300.1, 300.1…
## $ temperature <dbl> 272.1, 282.2, 285.2, 290.7, 292.7, 293.6, 294.6, 296.9, 297.8, 298.7…

Your turn

Filter to all ozone values between 320 and 325 in the first month of the last year

nasa %>% filter(ozone > 320 & ozone<325, month==first(month),
       year==last(year))
##OR
nasa %>% filter(between(ozone,320,325), month==first(month),
       year==last(year))

Slicing

Filtering by row number

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94
dat.1 %>% slice(1:4)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94
## 3   Control   P1    H    3 27.49 29.85
## 4   Control   P1    H    4 44.79 25.39
data.1 %>% slice(c(1:4,7))
## # A tibble: 5 x 7
##   Between Plot  Cond   Time  Temp   LAT  LONG
##   <fct>   <fct> <fct> <int> <dbl> <dbl> <dbl>
## 1 A1      P1    H         1  26.3  14.0  150.
## 2 A1      P1    M         2  27.9  12.9  145.
## 3 A1      P1    L         3  28.3  12.3  146.
## 4 A1      P2    H         4  28.4  12.1  145.
## 5 A2      P3    H         3  25.8  14.6  147.

Sampling

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94
dat.1 %>% sample_n(10, replace=TRUE)
##    Treatment Plot Dose Time  Resp1  Resp2
## 1    Control   P1    H    1   8.12   3.06
## 2  Exclusion   P4    H    4  82.29  60.15
## 3    Control   P1    M    2  37.54  17.62
## 4    Control   P2    L    3  90.22 113.87
## 5  Exclusion   P4    M    4 161.67 256.34
## 6    Control   P1    M    2  37.54  17.62
## 7  Exclusion   P4    H    4  82.29  60.15
## 8  Exclusion   P4    H    3  63.30  93.43
## 9    Control   P1    L    3  94.54 119.22
## 10   Control   P2    M    1  19.95  19.73

Sampling

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94
dat.1 %>% sample_frac(0.5, replace=TRUE)
##    Treatment Plot Dose Time  Resp1  Resp2
## 1  Exclusion   P3    L    4 238.76  54.23
## 2    Control   P2    H    1   8.14  23.93
## 3  Exclusion   P4    H    4  82.29  60.15
## 4  Exclusion   P3    H    1  21.86  23.58
## 5  Exclusion   P4    L    4 242.31 304.70
## 6  Exclusion   P3    H    1  21.86  23.58
## 7    Control   P2    L    3  90.22 113.87
## 8  Exclusion   P4    L    2 117.88 167.90
## 9    Control   P2    M    4  81.78 136.66
## 10 Exclusion   P4    H    3  63.30  93.43
## 11 Exclusion   P3    M    1  38.57  30.63
## 12   Control   P2    H    1   8.14  23.93
## 13   Control   P2    L    2  62.35 123.78
## 14 Exclusion   P3    M    1  38.57  30.63
## 15 Exclusion   P3    H    4  75.59  90.76
## 16 Exclusion   P4    M    4 161.67 256.34
## 17 Exclusion   P4    L    4 242.31 304.70
## 18 Exclusion   P3    L    1  61.16  39.53
## 19 Exclusion   P4    L    1  57.93  85.24
## 20   Control   P2    L    1  32.76  30.70
## 21 Exclusion   P3    M    3 124.08 124.09
## 22 Exclusion   P4    H    2  39.82  39.07
## 23   Control   P2    M    1  19.95  19.73
## 24   Control   P1    L    1  31.73  21.22

Effects of filtering

dat.1 %>% head(2)
##   Treatment Plot Dose Time Resp1 Resp2
## 1   Control   P1    H    1  8.12  3.06
## 2   Control   P1    H    2 20.55 25.94
#examine the levels of the Cond factor
levels(dat.1$Dose)
## [1] "H" "L" "M"

Effects of filtering

#subset the dataset to just Cond H
dat.3 <- dat.1 %>% filter(Plot=='P1')
#examine subset data
dat.3
##    Treatment Plot Dose Time  Resp1  Resp2
## 1    Control   P1    H    1   8.12   3.06
## 2    Control   P1    H    2  20.55  25.94
## 3    Control   P1    H    3  27.49  29.85
## 4    Control   P1    H    4  44.79  25.39
## 5    Control   P1    M    1  20.99  20.31
## 6    Control   P1    M    2  37.54  17.62
## 7    Control   P1    M    3  61.46  98.44
## 8    Control   P1    M    4  82.21 160.01
## 9    Control   P1    L    1  31.73  21.22
## 10   Control   P1    L    2  59.08  37.51
## 11   Control   P1    L    3  94.54 119.22
## 12   Control   P1    L    4 121.17 116.45
#examine the levels of the Cond factor
levels(dat.3$Dose)
## [1] "H" "L" "M"
levels(dat.3$Plot)
## [1] "P1" "P2" "P3" "P4"
levels(dat.3$Treatment)
## [1] "Control"   "Exclusion"

Effects of filtering

Correction - all factors

#subset the dataset to just Dose H
dat.3 <-  dat.1 %>% filter(Plot=='P1')
#drop the unused factor levels from all factors
dat.3 <- dat.3 %>% droplevels()
#examine the levels of each factor
levels(dat.3$Dose)
## [1] "H" "L" "M"
levels(dat.3$Plot)
## [1] "P1"
levels(dat.3$Treatment)
## [1] "Control"

Effects of filtering

Correction - single factor

#subset the dataset to just Dose H
dat.3 <- dat.1 %>% filter(Plot=='P1')
#drop the unused factor levels from Dose
dat.3 <- dat.3 %>% mutate(Plot=factor(Plot))
#examine the levels of each factor
levels(dat.3$Dose)
## [1] "H" "L" "M"
levels(dat.3$Plot)
## [1] "P1"
levels(dat.3$Treatment)
## [1] "Control"   "Exclusion"

Reshaping data




Reshaping data frames

Wide data

  Between Plot Time.0 Time.1 Time.2
R1 A1 P1 8 14 14
R2 A1 P2 10 12 11
R3 A2 P3 7 11 8
R4 A2 P4 11 9 2

Wide to long (melt)

data.w %>% pivot_longer(Time.0:Time.2,  names_to = 'Time',
                        values_to='Count')
## OR
data.w %>% pivot_longer(c(-Between, -Plot),  names_to = 'Time',
                        values_to='Count')
## # A tibble: 12 x 4
##    Between Plot  Time   Count
##    <fct>   <fct> <chr>  <int>
##  1 A1      P1    Time.0     8
##  2 A1      P1    Time.1    14
##  3 A1      P1    Time.2    14
##  4 A1      P2    Time.0    10
##  5 A1      P2    Time.1    12
##  6 A1      P2    Time.2    11
##  7 A2      P3    Time.0     7
##  8 A2      P3    Time.1    11
##  9 A2      P3    Time.2     8
## 10 A2      P4    Time.0    11
## 11 A2      P4    Time.1     9
## 12 A2      P4    Time.2     2
## # A tibble: 12 x 4
##    Between Plot  Time   Count
##    <fct>   <fct> <chr>  <int>
##  1 A1      P1    Time.0     8
##  2 A1      P1    Time.1    14
##  3 A1      P1    Time.2    14
##  4 A1      P2    Time.0    10
##  5 A1      P2    Time.1    12
##  6 A1      P2    Time.2    11
##  7 A2      P3    Time.0     7
##  8 A2      P3    Time.1    11
##  9 A2      P3    Time.2     8
## 10 A2      P4    Time.0    11
## 11 A2      P4    Time.1     9
## 12 A2      P4    Time.2     2

Reshaping data frames

Wide data

  Between Plot Time.0 Time.1 Time.2
R1 A1 P1 8 14 14
R2 A1 P2 10 12 11
R3 A2 P3 7 11 8
R4 A2 P4 11 9 2

Wide to long (melt)

## OR
data.w %>% pivot_longer(starts_with('Time'),  names_to = 'Time',
                        values_to='Count',
                        names_prefix='Time.')
## # A tibble: 12 x 4
##    Between Plot  Time  Count
##    <fct>   <fct> <chr> <int>
##  1 A1      P1    0         8
##  2 A1      P1    1        14
##  3 A1      P1    2        14
##  4 A1      P2    0        10
##  5 A1      P2    1        12
##  6 A1      P2    2        11
##  7 A2      P3    0         7
##  8 A2      P3    1        11
##  9 A2      P3    2         8
## 10 A2      P4    0        11
## 11 A2      P4    1         9
## 12 A2      P4    2         2

Reshaping data frames

Long data

Resp1 Resp2 Between Plot Subplot Within
8 17 A1 P1 S1 B1
10 18 A1 P1 S1 B2
7 17 A1 P1 S2 B1
11 21 A1 P1 S2 B2
14 19 A2 P2 S3 B1
12 13 A2 P2 S3 B2
11 24 A2 P2 S4 B1
9 18 A2 P2 S4 B2
14 25 A3 P3 S5 B1
11 18 A3 P3 S5 B2
8 27 A3 P3 S6 B1
2 22 A3 P3 S6 B2
8 17 A1 P4 S7 B1
10 22 A1 P4 S7 B2
7 16 A1 P4 S8 B1
12 13 A1 P4 S8 B2
11 23 A2 P5 S9 B1
12 19 A2 P5 S9 B2
12 23 A2 P5 S10 B1
10 21 A2 P5 S10 B2
3 17 A3 P6 S11 B1
11 16 A3 P6 S11 B2
13 26 A3 P6 S12 B1
7 28 A3 P6 S12 B2

Reshaping data frames

data %>% head(2)
##   Resp1 Resp2 Between Plot Subplot Within
## 1     8    17      A1   P1      S1     B1
## 2    10    18      A1   P1      S1     B2

Widen (cast)

Widen Resp1 for repeated measures (Within)

data %>% select(-Resp2) %>%
  pivot_wider(names_from=Within,  values_from=c(Resp1))
## # A tibble: 12 x 5
##    Between Plot  Subplot    B1    B2
##    <fct>   <fct> <fct>   <int> <int>
##  1 A1      P1    S1          8    10
##  2 A1      P1    S2          7    11
##  3 A2      P2    S3         14    12
##  4 A2      P2    S4         11     9
##  5 A3      P3    S5         14    11
##  6 A3      P3    S6          8     2
##  7 A1      P4    S7          8    10
##  8 A1      P4    S8          7    12
##  9 A2      P5    S9         11    12
## 10 A2      P5    S10        12    10
## 11 A3      P6    S11         3    11
## 12 A3      P6    S12        13     7

Reshaping data frames

Widen Resp1 and Resp2 for repeated measures (Within)

data %>% head(2)
##   Resp1 Resp2 Between Plot Subplot Within
## 1     8    17      A1   P1      S1     B1
## 2    10    18      A1   P1      S1     B2
data %>% pivot_wider(names_from=Within, values_from=c(Resp1, Resp2))
## # A tibble: 12 x 7
##    Between Plot  Subplot Resp1_B1 Resp1_B2 Resp2_B1 Resp2_B2
##    <fct>   <fct> <fct>      <int>    <int>    <int>    <int>
##  1 A1      P1    S1             8       10       17       18
##  2 A1      P1    S2             7       11       17       21
##  3 A2      P2    S3            14       12       19       13
##  4 A2      P2    S4            11        9       24       18
##  5 A3      P3    S5            14       11       25       18
##  6 A3      P3    S6             8        2       27       22
##  7 A1      P4    S7             8       10       17       22
##  8 A1      P4    S8             7       12       16       13
##  9 A2      P5    S9            11       12       23       19
## 10 A2      P5    S10           12       10       23       21
## 11 A3      P6    S11            3       11       17       16
## 12 A3      P6    S12           13        7       26       28

Combining data




Merging data frames

Bio data (missing Subplot 3)

  Resp1 Resp2 Between Plot Subplot
1 8 18 A1 P1 S1
2 10 21 A1 P1 S2
4 11 23 A1 P2 S4
5 14 22 A2 P3 S5
6 12 24 A2 P3 S6
7 11 23 A2 P4 S7
8 9 20 A2 P4 S8
9 14 11 A3 P5 S9
10 11 22 A3 P5 S10
11 8 24 A3 P6 S11
12 2 16 A3 P6 S12

Physio-chemical data (missing S7)

  Chem1 Chem2 Between Plot Subplot
1 1.453 0.8858 A1 P1 S1
2 3.266 0.18 A1 P1 S2
3 1.179 5.078 A1 P2 S3
4 13.4 1.576 A1 P2 S4
5 3.779 1.622 A2 P3 S5
6 1.197 4.237 A2 P3 S6
8 5.688 2.986 A2 P4 S8
9 4.835 4.133 A3 P5 S9
10 2.003 3.604 A3 P5 S10
11 12.33 1.776 A3 P6 S11
12 4.014 0.2255 A3 P6 S12

Merging data frames

Merge bio and chem data (only keep full matches - an inner join)

data.bio %>% inner_join(data.chem)
##    Resp1 Resp2 Between Plot Subplot     Chem1     Chem2
## 1      8    18      A1   P1      S1  1.452878 0.8858208
## 2     10    21      A1   P1      S2  3.266253 0.1800177
## 3     11    23      A1   P2      S4 13.400350 1.5762780
## 4     14    22      A2   P3      S5  3.779183 1.6222430
## 5     12    24      A2   P3      S6  1.196657 4.2369184
## 6      9    20      A2   P4      S8  5.687807 2.9859003
## 7     14    11      A3   P5      S9  4.834518 4.1328919
## 8     11    22      A3   P5     S10  2.002931 3.6043314
## 9      8    24      A3   P6     S11 12.326867 1.7763576
## 10     2    16      A3   P6     S12  4.014221 0.2255188
  • S3 and S7 absent

Merging data frames

Merge bio and chem data (keep all data - outer join)

data.bio %>% full_join(data.chem)
##    Resp1 Resp2 Between Plot Subplot     Chem1     Chem2
## 1      8    18      A1   P1      S1  1.452878 0.8858208
## 2     10    21      A1   P1      S2  3.266253 0.1800177
## 3     11    23      A1   P2      S4 13.400350 1.5762780
## 4     14    22      A2   P3      S5  3.779183 1.6222430
## 5     12    24      A2   P3      S6  1.196657 4.2369184
## 6     11    23      A2   P4      S7        NA        NA
## 7      9    20      A2   P4      S8  5.687807 2.9859003
## 8     14    11      A3   P5      S9  4.834518 4.1328919
## 9     11    22      A3   P5     S10  2.002931 3.6043314
## 10     8    24      A3   P6     S11 12.326867 1.7763576
## 11     2    16      A3   P6     S12  4.014221 0.2255188
## 12    NA    NA      A1   P2      S3  1.178652 5.0780682
  • note the order of Subplot

Merging data frames

Merge bio and chem data (only keep full BIO matches - left join)

data.bio %>% left_join(data.chem)
##    Resp1 Resp2 Between Plot Subplot     Chem1     Chem2
## 1      8    18      A1   P1      S1  1.452878 0.8858208
## 2     10    21      A1   P1      S2  3.266253 0.1800177
## 3     11    23      A1   P2      S4 13.400350 1.5762780
## 4     14    22      A2   P3      S5  3.779183 1.6222430
## 5     12    24      A2   P3      S6  1.196657 4.2369184
## 6     11    23      A2   P4      S7        NA        NA
## 7      9    20      A2   P4      S8  5.687807 2.9859003
## 8     14    11      A3   P5      S9  4.834518 4.1328919
## 9     11    22      A3   P5     S10  2.002931 3.6043314
## 10     8    24      A3   P6     S11 12.326867 1.7763576
## 11     2    16      A3   P6     S12  4.014221 0.2255188

Merging data frames

Merge bio and chem data (only keep full CHEM matches - right join)

data.bio %>% right_join(data.chem)
##    Resp1 Resp2 Between Plot Subplot     Chem1     Chem2
## 1      8    18      A1   P1      S1  1.452878 0.8858208
## 2     10    21      A1   P1      S2  3.266253 0.1800177
## 3     11    23      A1   P2      S4 13.400350 1.5762780
## 4     14    22      A2   P3      S5  3.779183 1.6222430
## 5     12    24      A2   P3      S6  1.196657 4.2369184
## 6      9    20      A2   P4      S8  5.687807 2.9859003
## 7     14    11      A3   P5      S9  4.834518 4.1328919
## 8     11    22      A3   P5     S10  2.002931 3.6043314
## 9      8    24      A3   P6     S11 12.326867 1.7763576
## 10     2    16      A3   P6     S12  4.014221 0.2255188
## 11    NA    NA      A1   P2      S3  1.178652 5.0780682

VLOOKUP

VLOOKUP

Biological data set (data.bio)

##    Resp1 Resp2 Between Plot Subplot
## 1      8    18      A1   P1      S1
## 2     10    21      A1   P1      S2
## 4     11    23      A1   P2      S4
## 5     14    22      A2   P3      S5
## 6     12    24      A2   P3      S6
## 7     11    23      A2   P4      S7
## 8      9    20      A2   P4      S8
## 9     14    11      A3   P5      S9
## 10    11    22      A3   P5     S10
## 11     8    24      A3   P6     S11
## 12     2    16      A3   P6     S12

Geographical data set (lookup table) (data.geo)

##   Plot     LAT     LONG
## 1   P1 17.9605 145.4326
## 2   P2 17.5210 146.1983
## 3   P3 17.0011 146.3839
## 4   P4 18.2350 146.7934
## 5   P5 18.9840 146.0345
## 6   P6 20.1154 146.4672

VLOOKUP

Incorporate (merge) the lat/longs into the bio data

data.bio %>% left_join(data.geo,by=c("Plot"))
##    Resp1 Resp2 Between Plot Subplot     LAT     LONG
## 1      8    18      A1   P1      S1 17.9605 145.4326
## 2     10    21      A1   P1      S2 17.9605 145.4326
## 3     11    23      A1   P2      S4 17.5210 146.1983
## 4     14    22      A2   P3      S5 17.0011 146.3839
## 5     12    24      A2   P3      S6 17.0011 146.3839
## 6     11    23      A2   P4      S7 18.2350 146.7934
## 7      9    20      A2   P4      S8 18.2350 146.7934
## 8     14    11      A3   P5      S9 18.9840 146.0345
## 9     11    22      A3   P5     S10 18.9840 146.0345
## 10     8    24      A3   P6     S11 20.1154 146.4672
## 11     2    16      A3   P6     S12 20.1154 146.4672

Applied examples

Tikus Island coral data

##     Psammocora contigua Psammocora digitata time rep
## V1                    0                   0   81   1
## V2                    0                   0   81   2
## V3                    0                   0   81   3
## V4                    0                   0   81   4
## V5                    0                   0   81   5
## V6                    0                   0   81   6
## V7                    0                   0   81   7
## V8                    0                   0   81   8
## V9                    0                   0   81   9
## V10                   0                   0   81  10
## Rows: 60
## Columns: 77
## $ `Psammocora contigua`    <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
## $ `Psammocora digitata`    <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
## $ `Pocillopora damicornis` <int> 79, 51, 42, 15, 9, 72, 0, 16, 0, 16…
## $ `Pocillopora verrucosa`  <int> 32, 21, 35, 0, 0, 0, 41, 25, 38, 0,…
## $ `Stylopora pistillata`   <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
## $ `Acropora bruegemanni`   <int> 0, 44, 0, 11, 9, 10, 0, 0, 0, 37, 0…
## $ `Acropora robusta`       <int> 0, 35, 40, 0, 0, 0, 0, 0, 0, 0, 0, …
## $ `Acropora grandis`       <int> 0, 0, 0, 0, 0, 0, 60, 0, 0, 0, 0, 0…
## $ `Acropora intermedia`    <int> 30, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0…
## $ `Acropora formosa`       <int> 75, 0, 15, 0, 125, 0, 0, 0, 10, 0, …
## $ `Acropora splendida`     <int> 0, 22, 0, 31, 0, 9, 16, 0, 0, 20, 0…
## $ `Acropera aspera`        <int> 17, 18, 9, 8, 23, 0, 17, 13, 16, 18…
## $ `Acropora hyacinthus`    <int> 141, 34, 55, 54, 0, 0, 0, 0, 0, 0, …
## $ `Acropora palifera`      <int> 32, 0, 44, 0, 17, 0, 0, 0, 0, 0, 0,…
## $ `Acropora cytherea`      <int> 108, 33, 14, 122, 0, 0, 0, 8, 0, 0,…
## $ `Acropora tenuis`        <int> 0, 25, 0, 0, 0, 22, 28, 0, 0, 0, 0,…
## $ `Acropora pulchra`       <int> 0, 0, 15, 52, 62, 33, 0, 0, 24, 0, …
## $ `Acropora nasuta`        <int> 43, 21, 19, 0, 0, 0, 10, 0, 0, 0, 0…
## $ `Acropora humilis`       <int> 31, 25, 0, 19, 0, 0, 0, 0, 0, 0, 0,…
## $ `Acropora diversa`       <int> 22, 19, 20, 13, 23, 14, 0, 12, 12, …
## $ `Acropora digitifera`    <int> 30, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0…
## $ `Acropora divaricata`    <int> 0, 32, 55, 0, 0, 0, 0, 0, 0, 0, 0, …
## $ `Acropora subglabra`     <int> 51, 0, 0, 44, 15, 0, 0, 25, 0, 0, 0…
## $ `Acropora cerealis`      <int> 0, 75, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0…
## $ `Acropora valida`        <int> 0, 0, 0, 30, 0, 0, 0, 0, 0, 0, 0, 0…
## $ `Acropora acuminata`     <int> 20, 0, 71, 0, 15, 0, 25, 25, 0, 0, …
## $ `Acropora elsevi`        <int> 30, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0…
## $ `Acropora millepora`     <int> 17, 14, 0, 20, 0, 0, 0, 0, 0, 0, 0,…
## $ `Montipora monasteriata` <int> 60, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0…
## $ `Montipora tuberculosa`  <int> 0, 15, 15, 0, 0, 0, 0, 0, 0, 0, 0, …
## $ `Montipora hispida`      <int> 0, 0, 0, 32, 40, 24, 0, 0, 0, 0, 0,…
## $ `Montipora digitata`     <int> 0, 0, 0, 0, 0, 77, 84, 53, 71, 351,…
## $ `Montipora foliosa`      <int> 0, 0, 0, 0, 50, 71, 62, 81, 24, 0, …
## $ `Montipora verrucosa`    <int> 0, 0, 30, 0, 0, 0, 0, 0, 0, 0, 0, 0…
## $ `Fungia fungites`        <int> 0, 0, 18, 17, 0, 0, 0, 0, 0, 0, 0, …
## $ `Fungia paumotensis`     <int> 0, 33, 0, 0, 0, 0, 0, 0, 0, 0, 12, …
## $ `Fungia concina`         <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10…
## $ `Fungia scutaria`        <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
## $ `Halomitra limax`        <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
## $ `Pavona varians`         <int> 30, 0, 0, 0, 0, 0, 0, 0, 0, 0, 30, …
## $ `Pavona venosa`          <int> 0, 24, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0…
## $ `Pavona cactus`          <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
## $ `Coeloseris mayeri`      <int> 20, 0, 15, 0, 9, 19, 0, 0, 25, 0, 0…
## $ `Galaxea fascicularis`   <int> 51, 27, 31, 24, 0, 13, 0, 0, 0, 0, …
## $ `Symphyllia radians`     <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
## $ `Lobophyllia corymbosa`  <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 12…
## $ `Lobophyllia hemprichii` <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
## $ `Porites cylindrica`     <int> 61, 24, 0, 20, 0, 0, 0, 0, 0, 0, 10…
## $ `Porites lichen`         <int> 0, 47, 49, 0, 0, 0, 0, 0, 0, 0, 0, …
## $ `Porites lobata`         <int> 36, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0…
## $ `Porites lutea`          <int> 30, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0…
## $ `Porites nigrescens`     <int> 0, 0, 0, 21, 0, 9, 25, 0, 45, 26, 0…
## $ `Porites solida`         <int> 0, 0, 10, 0, 17, 0, 31, 41, 0, 0, 0…
## $ `Porites stephensoni`    <int> 0, 0, 0, 0, 0, 0, 0, 30, 0, 0, 0, 0…
## $ `Goniopora lobata`       <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
## $ `Favia pallida`          <int> 10, 20, 0, 0, 0, 0, 0, 0, 0, 0, 0, …
## $ `Favia speciosa`         <int> 0, 0, 30, 0, 0, 0, 0, 0, 0, 0, 0, 0…
## $ `Favia stelligera`       <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
## $ `Favia rotumana`         <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
## $ `Favites abdita`         <int> 33, 41, 23, 27, 91, 63, 72, 48, 71,…
## $ `Favites chinensis`      <int> 0, 44, 78, 61, 44, 0, 55, 30, 30, 0…
## $ `Goniastrea rectiformis` <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 6, 0,…
## $ `Goniastrea pectinata`   <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
## $ `Goniastrea sp`          <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
## $ `Dulophyllia crispa`     <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 9,…
## $ `Platygyra daedalea`     <int> 0, 27, 55, 0, 71, 74, 55, 48, 0, 0,…
## $ `Platygyra sinensis`     <int> 47, 27, 56, 26, 0, 0, 0, 0, 0, 0, 0…
## $ `Hydnopora rigida`       <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
## $ `Leptastrea purpurea`    <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
## $ `Leptastrea pruinosa`    <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
## $ `Cyphastrea serailia`    <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 19, 0…
## $ `Millepora platyphylla`  <int> 30, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0…
## $ `Millepora dichotoma`    <int> 21, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0…
## $ `Millepora intrincata`   <int> 24, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0…
## $ `Heliopora coerulea`     <int> 461, 271, 221, 154, 0, 0, 0, 0, 0, …
## $ time                     <fct> 81, 81, 81, 81, 81, 81, 81, 81, 81,…
## $ rep                      <fct> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 1, 2…

Tikus Island coral data

Explore/Process data

  • Convert abundance to cover
  • Mean cover of total Acropora per year
  • NOTE there is a typo ‘Acropera’

Tikus Island coral data

Explore/Process data

  • Convert abundance to cover (abundance is the length in cm of a 10m transect containing the species)
  • Mean cover of total Acropora per year
  • NOTE there is a typo ‘Acropera’

Step 1. fix typo (rename) - backticks

tikus %>% rename(`Acropora aspera`=`Acropera aspera`)

Tikus Island coral data

Explore/Process data

  • Convert abundance to cover
  • Mean cover of total Acropora per year
  • NOTE there is a typo ‘Acropera’

Step 2. melt data (gather)

tikus %>% rename(`Acropora aspera`=`Acropera aspera`) %>%
    gather(Species, Abundance,-time,-rep)

Tikus Island coral data

Explore/Process data

  • Convert abundance to cover
  • Mean cover of total Acropora per year
  • NOTE there is a typo ‘Acropera’

Step 3. Calculate Cover (mutate) (Abundance/10)

tikus %>% rename(`Acropora aspera`=`Acropera aspera`) %>%
    gather(Species, Abundance,-time,-rep) %>%
    mutate(Cover=Abundance/10)

Tikus Island coral data

Explore/Process data

  • Convert abundance to cover
  • Mean cover of total Acropora per year
  • NOTE there is a typo ‘Acropera’

Step 4. Split species into Genera and Species (separate)

tikus %>% rename(`Acropora aspera`=`Acropera aspera`) %>%
    gather(Species, Abundance,-time,-rep) %>%
    mutate(Cover=Abundance/10) %>%
    separate(Species,c('Genera','Species'))

Tikus Island coral data

Explore/Process data

  • Convert abundance to cover
  • Mean cover of total Acropora per year
  • NOTE there is a typo ‘Acropera’

Step 5. Subset just ‘Acropora’ (filter)

tikus %>% rename(`Acropora aspera`=`Acropera aspera`) %>%
    gather(Species, Abundance,-time,-rep) %>%
    mutate(Cover=Abundance/10) %>%
    separate(Species,c('Genera','Species')) %>%
    filter(Genera=='Acropora')

Tikus Island coral data

Explore/Process data

  • Convert abundance to cover
  • Mean cover of total Acropora per year
  • NOTE there is a typo ‘Acropera’

Step 6. Sum over all Species (group_by and summarise)

tikus %>% rename(`Acropora aspera`=`Acropera aspera`) %>%
    gather(Species, Abundance,-time,-rep) %>%
    mutate(Cover=Abundance/10) %>%
    separate(Species,c('Genera','Species')) %>%
    filter(Genera=='Acropora') %>%
    group_by(time,rep) %>%
    summarise(SumCover=sum(Cover))

Tikus Island coral data

Explore/Process data

  • Convert abundance to cover
  • Mean cover of total Acropora per year
  • NOTE there is a typo ‘Acropera’

Step 7. Summarise per year

tikus %>% rename(`Acropora aspera`=`Acropera aspera`) %>%
    gather(Species, Abundance,-time,-rep) %>%
    mutate(Cover=Abundance/10) %>%
    separate(Species,c('Genera','Species')) %>%
    filter(Genera=='Acropora') %>%
    group_by(time,rep) %>%
    summarise(SumCover=sum(Cover)) %>%
    group_by(time) %>%
    summarise(Mean=mean(SumCover),
              Var=var(SumCover))
## # A tibble: 6 x 3
##   time   Mean   Var
## * <fct> <dbl> <dbl>
## 1 81    25.6  383. 
## 2 83     0      0  
## 3 84     0      0  
## 4 85     2.43  14.2
## 5 87     8.01  68.5
## 6 88     8.55 106.

Tikus Island coral data

tikus %>% rename(`Acropora aspera`=`Acropera aspera`) %>%
    gather(Species, Abundance,-time,-rep) %>%
    mutate(Cover=Abundance/10) %>%
    separate(Species,c('Genera','Species')) %>%
    filter(Genera=='Acropora') %>%
    group_by(time,rep) %>%
    summarise(SumCover=sum(Cover)) %>%
    group_by(time) %>%
    summarise(Mean=mean(SumCover),
              Var=var(SumCover))
## # A tibble: 6 x 3
##   time   Mean   Var
## * <fct> <dbl> <dbl>
## 1 81    25.6  383. 
## 2 83     0      0  
## 3 84     0      0  
## 4 85     2.43  14.2
## 5 87     8.01  68.5
## 6 88     8.55 106.

Tikus Island coral data

Can you modify so that we get the means and var for each Genera per year?

tikus %>% rename(`Acropora aspera`=`Acropera aspera`) %>%
    gather(Species, Abundance,-time,-rep) %>%
    mutate(Cover=Abundance/10) %>%
    separate(Species,c('Genera','Species')) %>%
    group_by(time,rep,Genera) %>%
    summarise(SumCover=sum(Cover)) %>%
    group_by(time,Genera) %>%
    summarise(Mean=mean(SumCover),
              Var=var(SumCover))
## # A tibble: 144 x 4
## # Groups:   time [6]
##    time  Genera       Mean    Var
##    <fct> <chr>       <dbl>  <dbl>
##  1 81    Acropora    25.6  383.  
##  2 81    Coeloseris   0.88   1.02
##  3 81    Cyphastrea   0      0   
##  4 81    Dulophyllia  0      0   
##  5 81    Favia        0.6    1.16
##  6 81    Favites      8.22  14.9 
##  7 81    Fungia       0.68   1.38
##  8 81    Galaxea      1.46   3.23
##  9 81    Goniastrea   0      0   
## 10 81    Goniopora    0      0   
## # … with 134 more rows

Tikus Island coral data

What about the means and var for the top 3 Genera per year (sorted from highest to lowest)?

tikus %>% rename(`Acropora aspera`=`Acropera aspera`) %>%
    gather(Species, Abundance,-time,-rep) %>%
    mutate(Cover=Abundance/10) %>%
    separate(Species,c('Genera','Species')) %>%
    group_by(time,rep,Genera) %>%
    summarise(SumCover=sum(Cover)) %>%
    group_by(time,Genera) %>%
    summarise(Mean=mean(SumCover),
              Var=var(SumCover)) %>%
    top_n(3,Mean) %>%
    arrange(desc(Mean))
## # A tibble: 18 x 4
## # Groups:   time [6]
##    time  Genera     Mean    Var
##    <fct> <chr>     <dbl>  <dbl>
##  1 87    Montipora 27.4  966.  
##  2 81    Acropora  25.6  383.  
##  3 85    Montipora 20.5  171.  
##  4 85    Porites   19.0   51.3 
##  5 88    Montipora 11.8  644.  
##  6 81    Montipora 11.4   95.7 
##  7 81    Heliopora 11.1  262.  
##  8 84    Montipora 11.0   70.5 
##  9 88    Porites    9.84  41.4 
## 10 88    Acropora   8.55 106.  
## 11 87    Acropora   8.01  68.5 
## 12 87    Porites    4.49  35.8 
## 13 84    Porites    2.94   6.65
## 14 85    Platygyra  2.55   8.74
## 15 83    Porites    1.74   2.07
## 16 84    Pavona     1.2    3.33
## 17 83    Fungia     1.14   3.64
## 18 83    Montipora  0.93   1.57